[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:28 dist-test-slave-3986 krb5kdc[15082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:28 dist-test-slave-3986 krb5kdc[15082](info): set up 2 sockets
Sep 26 22:44:28 dist-test-slave-3986 krb5kdc[15082](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 22:44:29.522531 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:29 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:29.635541 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44831
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:34813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:44831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:29.714442 15095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:29.714673 15095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:29.714748 15095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:29.715896 15095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:29.715960 15095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:29.715986 15095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:29.716017 15095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:29.716040 15095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:29.717497 15095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:34813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:44831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:29.717769 15095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:29.717995 15095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:29.722703 15101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:29.722714 15100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:29.722759 15103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:29.722961 15095 server_base.cc:1034] running on GCE node
I20260926 22:44:29.723248 15095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:29.723528 15095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:29.724701 15095 hybrid_clock.cc:648] HybridClock initialized: now 1790462669724679 us; error 40 us; skew 500 ppm
Sep 26 22:44:29 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462669, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:29.727109 15095 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:29.727535 15095 webserver.cc:460] Webserver started at http://127.14.184.254:40539/ using document root <none> and password file <none>
I20260926 22:44:29.727758 15095 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:29.727824 15095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:29.727988 15095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:29.728987 15095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "ffee0e7fca324641bc9e2eee02ac8603"
format_stamp: "Formatted at 2026-09-26 22:44:29 on dist-test-slave-3986"
server_key: "d03e4b1e67596aa44930918d36c58c20"
server_key_iv: "8ed936fb5b7dcdfefd28d3740fa1b44b"
server_key_version: "encryptionkey@0"
I20260926 22:44:29.729377 15095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "ffee0e7fca324641bc9e2eee02ac8603"
format_stamp: "Formatted at 2026-09-26 22:44:29 on dist-test-slave-3986"
server_key: "d03e4b1e67596aa44930918d36c58c20"
server_key_iv: "8ed936fb5b7dcdfefd28d3740fa1b44b"
server_key_version: "encryptionkey@0"
I20260926 22:44:29.730664 15095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:44:29.731446 15110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:29.731639 15095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:29.731729 15095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "ffee0e7fca324641bc9e2eee02ac8603"
format_stamp: "Formatted at 2026-09-26 22:44:29 on dist-test-slave-3986"
server_key: "d03e4b1e67596aa44930918d36c58c20"
server_key_iv: "8ed936fb5b7dcdfefd28d3740fa1b44b"
server_key_version: "encryptionkey@0"
I20260926 22:44:29.731798 15095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:29.760658 15095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:29.763051 15095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:29.763195 15095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:29.767033 15095 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:44831
I20260926 22:44:29.767107 15162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:44831 every 8 connection(s)
I20260926 22:44:29.767438 15095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:29.768337 15163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:29.770040 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15095
I20260926 22:44:29.770117 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:29.770596 15163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603: Bootstrap starting.
I20260926 22:44:29.770820 15075 external_mini_cluster.cc:1428] Setting key fa1461344d73408e631abba71cefa60a
I20260926 22:44:29.771271 15163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:29.771623 15163 log.cc:824] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:29.772449 15163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603: No bootstrap required, opened a new log
I20260926 22:44:29.774292 15163 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffee0e7fca324641bc9e2eee02ac8603" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44831 } }
I20260926 22:44:29.774493 15163 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:29.774529 15163 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffee0e7fca324641bc9e2eee02ac8603, State: Initialized, Role: FOLLOWER
I20260926 22:44:29.774646 15163 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffee0e7fca324641bc9e2eee02ac8603" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44831 } }
I20260926 22:44:29.774773 15163 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:29.774814 15163 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:29.774883 15163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:29.775395 15163 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffee0e7fca324641bc9e2eee02ac8603" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44831 } }
I20260926 22:44:29.775480 15163 leader_election.cc:302] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffee0e7fca324641bc9e2eee02ac8603; no voters:
I20260926 22:44:29.775694 15163 leader_election.cc:288] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:29.775753 15168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:29.775913 15168 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [term 1 LEADER]: Becoming Leader. State: Replica: ffee0e7fca324641bc9e2eee02ac8603, State: Running, Role: LEADER
I20260926 22:44:29.775985 15163 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:29.776013 15168 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffee0e7fca324641bc9e2eee02ac8603" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44831 } }
I20260926 22:44:29.776356 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffee0e7fca324641bc9e2eee02ac8603. Latest consensus state: current_term: 1 leader_uuid: "ffee0e7fca324641bc9e2eee02ac8603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffee0e7fca324641bc9e2eee02ac8603" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44831 } } }
I20260926 22:44:29.776443 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:29.776368 15170 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffee0e7fca324641bc9e2eee02ac8603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffee0e7fca324641bc9e2eee02ac8603" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44831 } } }
I20260926 22:44:29.776641 15170 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603 [sys.catalog]: This master's current role is: LEADER
W20260926 22:44:29.777776 15183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:44:29.777837 15183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:44:29.778628 15175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:29.778847 15175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:29.781143 15175 catalog_manager.cc:1347] Generated new cluster ID: 710864466ce24c738ad3e1faf1f1298d
I20260926 22:44:29.781198 15175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:29 dist-test-slave-3986 krb5kdc[15082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:29.789839 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:29.771705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.14.184.254:44831)
0926 22:44:29.771862 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:29.771872 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:29.772114 (+ 242us) server_negotiation.cc:407] Connection header received
0926 22:44:29.774486 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:29.774495 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:29.774554 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:29.774742 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:29.778090 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:29.779479 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:29.780550 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:29.781171 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:29.781279 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:29.785675 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:29.785679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:29.785679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:29.785690 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:29.787677 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:29.788187 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:29.788189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:29.788190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:29.788291 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:29.788579 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:29.788579 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:29.788580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:29.788862 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:29.788997 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:29.789312 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 22:44:29.789438 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":86,"threads_started":1}
I20260926 22:44:29.822625 15175 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:29.823385 15175 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:29.826649 15175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffee0e7fca324641bc9e2eee02ac8603: Generated new TSK 0
I20260926 22:44:29.826864 15175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:29.858493 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:44831
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:34813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:29.948737 15189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:29.948964 15189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:29.949005 15189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:29.950163 15189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:29.950237 15189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:29.950282 15189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:29.951815 15189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:34813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:44831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:29.952075 15189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:29.952308 15189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:29.957399 15197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:29.957379 15195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:29.957379 15194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:29.957404 15189 server_base.cc:1034] running on GCE node
I20260926 22:44:29.957825 15189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:29.958079 15189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:29.959237 15189 hybrid_clock.cc:648] HybridClock initialized: now 1790462669959232 us; error 35 us; skew 500 ppm
Sep 26 22:44:29 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462669, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:29.961623 15189 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:29.962036 15189 webserver.cc:460] Webserver started at http://127.14.184.193:34671/ using document root <none> and password file <none>
I20260926 22:44:29.962275 15189 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:29.962342 15189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:29.962482 15189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:29.963581 15189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "d36abe81befb4e96bb4e0a68e06c445e"
format_stamp: "Formatted at 2026-09-26 22:44:29 on dist-test-slave-3986"
server_key: "eb67c5237363a271a9a36694e90b6573"
server_key_iv: "01a18f804de14e3093dd93adbf79fd9f"
server_key_version: "encryptionkey@0"
I20260926 22:44:29.963943 15189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "d36abe81befb4e96bb4e0a68e06c445e"
format_stamp: "Formatted at 2026-09-26 22:44:29 on dist-test-slave-3986"
server_key: "eb67c5237363a271a9a36694e90b6573"
server_key_iv: "01a18f804de14e3093dd93adbf79fd9f"
server_key_version: "encryptionkey@0"
I20260926 22:44:29.965301 15189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:29.966022 15204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:29.966192 15189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:29.966287 15189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "d36abe81befb4e96bb4e0a68e06c445e"
format_stamp: "Formatted at 2026-09-26 22:44:29 on dist-test-slave-3986"
server_key: "eb67c5237363a271a9a36694e90b6573"
server_key_iv: "01a18f804de14e3093dd93adbf79fd9f"
server_key_version: "encryptionkey@0"
I20260926 22:44:29.966369 15189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:29.983355 15189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:29.985126 15189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:29.985293 15189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:29.985540 15189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:29.985882 15189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:29.985936 15189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:29.985970 15189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:29.985996 15189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:29.991945 15189 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:37951
I20260926 22:44:29.991993 15317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:37951 every 8 connection(s)
I20260926 22:44:29.992300 15189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:29.993441 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15189
I20260926 22:44:29.993541 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:29.993705 15075 external_mini_cluster.cc:1428] Setting key c14def095949885b83894cbec3214f59
Sep 26 22:44:29 dist-test-slave-3986 krb5kdc[15082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462669, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
I20260926 22:44:30.002794 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:29.993048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:44909 (local address 127.14.184.254:44831)
0926 22:44:29.993114 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:29.993116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:29.993291 (+ 175us) server_negotiation.cc:407] Connection header received
0926 22:44:29.993887 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:29.993889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:29.993902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:29.993950 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:29.995027 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:29.995655 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:29.996822 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:29.997015 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:29.997069 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:29.999645 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:29.999647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:29.999647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:29.999655 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:30.001179 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.001765 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.001767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.001767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.001824 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.002163 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.002164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.002164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.002272 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:30.002351 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:30.002611 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 22:44:30.002731 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:44:30.002954 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:29.993004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44831 (local address 127.14.184.193:44909)
0926 22:44:29.993151 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:29.993204 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:29.993729 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:29.994029 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:29.994034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:29.994735 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:29.994932 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:29.994937 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:29.995764 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:29.995765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:29.996734 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:29.996737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:29.997105 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:29.997106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:29.997259 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:29.997886 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:29.997896 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:29.999550 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:30.001264 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:30.001271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:30.001273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:30.001627 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:30.001896 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:30.001897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:30.001897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:30.002046 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:30.002368 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:30.002375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:30.002495 (+ 120us) client_negotiation.cc:769] Sending connection context
0926 22:44:30.002582 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 22:44:30.002761 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260926 22:44:30.003558 15318 heartbeater.cc:347] Connected to a master server at 127.14.184.254:44831
I20260926 22:44:30.003657 15318 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:30.003842 15318 heartbeater.cc:511] Master 127.14.184.254:44831 requested a full tablet report, sending...
I20260926 22:44:30.004348 15127 ts_manager.cc:194] Registered new tserver with Master: d36abe81befb4e96bb4e0a68e06c445e (127.14.184.193:37951)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
I20260926 22:44:30.005018 15127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:44909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:30.019516 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:44831
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:34813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:30.098816 15323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:30.099050 15323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:30.099086 15323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:30.100226 15323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:30.100302 15323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:30.100347 15323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:30.101846 15323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:34813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:44831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:30.102116 15323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:30.102347 15323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:30.107532 15329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:30.107646 15323 server_base.cc:1034] running on GCE node
W20260926 22:44:30.107545 15331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:30.107545 15328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:30.107980 15323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:30.108259 15323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:30.109421 15323 hybrid_clock.cc:648] HybridClock initialized: now 1790462670109419 us; error 34 us; skew 500 ppm
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:30.111905 15323 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:30.112319 15323 webserver.cc:460] Webserver started at http://127.14.184.194:41565/ using document root <none> and password file <none>
I20260926 22:44:30.112542 15323 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:30.112608 15323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:30.112757 15323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:30.113849 15323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "d88290768be6423d892e83f6ca7d814a"
format_stamp: "Formatted at 2026-09-26 22:44:30 on dist-test-slave-3986"
server_key: "238b7842ab9b74c94e53b1d40f4fd532"
server_key_iv: "0eaf1d1eb143067a1b4bea837df65d31"
server_key_version: "encryptionkey@0"
I20260926 22:44:30.114209 15323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "d88290768be6423d892e83f6ca7d814a"
format_stamp: "Formatted at 2026-09-26 22:44:30 on dist-test-slave-3986"
server_key: "238b7842ab9b74c94e53b1d40f4fd532"
server_key_iv: "0eaf1d1eb143067a1b4bea837df65d31"
server_key_version: "encryptionkey@0"
I20260926 22:44:30.115594 15323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:44:30.116340 15338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.116513 15323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.116617 15323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "d88290768be6423d892e83f6ca7d814a"
format_stamp: "Formatted at 2026-09-26 22:44:30 on dist-test-slave-3986"
server_key: "238b7842ab9b74c94e53b1d40f4fd532"
server_key_iv: "0eaf1d1eb143067a1b4bea837df65d31"
server_key_version: "encryptionkey@0"
I20260926 22:44:30.116689 15323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:30.135517 15323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:30.137377 15323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:30.137540 15323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:30.137773 15323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:30.138099 15323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:30.138156 15323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.138190 15323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:30.138218 15323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.144027 15323 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:35487
I20260926 22:44:30.144084 15451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:35487 every 8 connection(s)
I20260926 22:44:30.144518 15323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:30.153271 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:30.145323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:49665 (local address 127.14.184.254:44831)
0926 22:44:30.145377 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:30.145379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:30.145555 (+ 176us) server_negotiation.cc:407] Connection header received
0926 22:44:30.146070 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:30.146072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:30.146084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:30.146142 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:30.147105 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.147605 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.148614 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.148820 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.148906 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:30.150943 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:30.150944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:30.150945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:30.150951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:30.152120 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.152497 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.152498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.152499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.152546 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.152766 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.152766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.152766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.152875 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:30.152925 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:30.153149 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 22:44:30.153208 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:30.153450 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:30.145262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44831 (local address 127.14.184.194:49665)
0926 22:44:30.145426 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:30.145469 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:30.145962 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:30.146210 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:30.146214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:30.146810 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:30.147002 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:30.147006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.147711 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.147712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:30.148518 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:30.148520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.148917 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.148918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:30.149046 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:30.149474 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:30.149483 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:30.150862 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:30.152188 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:30.152194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:30.152195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:30.152418 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:30.152604 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:30.152604 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:30.152604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:30.152713 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:30.152929 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:30.152933 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:30.153042 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 22:44:30.153122 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 22:44:30.153235 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260926 22:44:30.153988 15452 heartbeater.cc:347] Connected to a master server at 127.14.184.254:44831
I20260926 22:44:30.154004 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15323
I20260926 22:44:30.154085 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:30.154093 15452 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:30.154275 15075 external_mini_cluster.cc:1428] Setting key 09a1526881b15ee364799bfe2565ff18
I20260926 22:44:30.154275 15452 heartbeater.cc:511] Master 127.14.184.254:44831 requested a full tablet report, sending...
I20260926 22:44:30.154718 15126 ts_manager.cc:194] Registered new tserver with Master: d88290768be6423d892e83f6ca7d814a (127.14.184.194:35487)
I20260926 22:44:30.155210 15126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:49665
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:30.177666 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:44831
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:34813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:30.257601 15457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:30.257833 15457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:30.257872 15457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:30.259076 15457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:30.259138 15457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:30.259184 15457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:30.260623 15457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:34813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:44831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:30.260878 15457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:30.261094 15457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:30.266047 15462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:30.266059 15463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:30.266064 15457 server_base.cc:1034] running on GCE node
W20260926 22:44:30.266043 15465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:30.266475 15457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:30.266772 15457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:30.267918 15457 hybrid_clock.cc:648] HybridClock initialized: now 1790462670267907 us; error 29 us; skew 500 ppm
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:30.270545 15457 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:30.271005 15457 webserver.cc:460] Webserver started at http://127.14.184.195:35423/ using document root <none> and password file <none>
I20260926 22:44:30.271225 15457 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:30.271271 15457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:30.271418 15457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:30.272469 15457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "15aa3844bb4a4249acb79917eaafc03b"
format_stamp: "Formatted at 2026-09-26 22:44:30 on dist-test-slave-3986"
server_key: "42dc5a0c1f38e27a7cc0f8bfede5a42f"
server_key_iv: "df44543459e6a18993a5e61345d07606"
server_key_version: "encryptionkey@0"
I20260926 22:44:30.272820 15457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "15aa3844bb4a4249acb79917eaafc03b"
format_stamp: "Formatted at 2026-09-26 22:44:30 on dist-test-slave-3986"
server_key: "42dc5a0c1f38e27a7cc0f8bfede5a42f"
server_key_iv: "df44543459e6a18993a5e61345d07606"
server_key_version: "encryptionkey@0"
I20260926 22:44:30.274112 15457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:44:30.275022 15472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.275213 15457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:30.275305 15457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "15aa3844bb4a4249acb79917eaafc03b"
format_stamp: "Formatted at 2026-09-26 22:44:30 on dist-test-slave-3986"
server_key: "42dc5a0c1f38e27a7cc0f8bfede5a42f"
server_key_iv: "df44543459e6a18993a5e61345d07606"
server_key_version: "encryptionkey@0"
I20260926 22:44:30.275367 15457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:30.289521 15457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:30.291806 15457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:30.291963 15457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:30.292176 15457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:30.292483 15457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:30.292516 15457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.292557 15457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:30.292584 15457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:30.298206 15457 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:37475
I20260926 22:44:30.298264 15585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:37475 every 8 connection(s)
I20260926 22:44:30.298571 15457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:30.301996 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15457
I20260926 22:44:30.302085 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:30.302227 15075 external_mini_cluster.cc:1428] Setting key 68f670263512c85056ead295c7cf8e05
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:30.307924 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:30.299377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:56799 (local address 127.14.184.254:44831)
0926 22:44:30.299445 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:30.299447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:30.299658 (+ 211us) server_negotiation.cc:407] Connection header received
0926 22:44:30.300142 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:30.300143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:30.300153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:30.300209 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:30.301074 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.301576 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.303031 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.303283 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.303359 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:30.305294 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:30.305296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:30.305296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:30.305304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:30.306832 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.307172 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.307172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.307173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.307224 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.307429 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.307429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.307429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.307526 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:30.307576 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:30.307790 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 22:44:30.307862 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:44:30.308104 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:30.299346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44831 (local address 127.14.184.195:56799)
0926 22:44:30.299529 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:30.299571 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:30.300056 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:30.300265 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:30.300269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:30.300813 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:30.301001 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:30.301005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.301660 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.301661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:30.302915 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:30.302918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.303376 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.303377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:30.303504 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:30.303926 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:30.303933 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:30.305209 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:30.306899 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:30.306904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:30.306906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:30.307108 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:30.307279 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:30.307280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:30.307280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:30.307376 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:30.307578 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:30.307582 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:30.307685 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 22:44:30.307764 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 22:44:30.307880 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260926 22:44:30.308585 15586 heartbeater.cc:347] Connected to a master server at 127.14.184.254:44831
I20260926 22:44:30.308662 15586 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:30.308799 15586 heartbeater.cc:511] Master 127.14.184.254:44831 requested a full tablet report, sending...
I20260926 22:44:30.309114 15126 ts_manager.cc:194] Registered new tserver with Master: 15aa3844bb4a4249acb79917eaafc03b (127.14.184.195:37475)
I20260926 22:44:30.309492 15126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:56799
I20260926 22:44:30.314210 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:30.336323 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:30.329071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.14.184.193:37951)
0926 22:44:30.329230 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:30.329242 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:30.329254 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:44:30.329279 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:30.329281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:30.329295 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:30.329365 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:30.329877 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.330572 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.331290 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.331601 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.331657 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:30.333676 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:30.333684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:30.333684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:30.333690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:30.335416 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.335709 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.335712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.335712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.335769 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.335917 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.335917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.335918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.336030 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:30.336132 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:30.336184 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 22:44:30.336243 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
W20260926 22:44:30.336774 15232 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:30.357159 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:30.350884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59084 (local address 127.14.184.193:37951)
0926 22:44:30.350939 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:30.350943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:30.351028 (+ 85us) server_negotiation.cc:407] Connection header received
0926 22:44:30.351069 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:30.351069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:30.351077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:30.351126 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:30.351605 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.352056 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.352715 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:30.352909 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:30.352958 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:30.355038 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:30.355039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:30.355040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:30.355046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:30.356367 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.356633 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.356634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.356634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.356679 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:30.356822 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:30.356822 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:30.356822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:30.356896 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:30.356975 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:30.357040 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 22:44:30.357097 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:44:30.357654 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15189
I20260926 22:44:30.362018 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15323
I20260926 22:44:30.366307 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15457
I20260926 22:44:30.370399 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15095
2026-09-26T22:44:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1893 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15600](info): set up 2 sockets
Sep 26 22:44:30 dist-test-slave-3986 krb5kdc[15600](info): commencing operation
krb5kdc: starting...
W20260926 22:44:31.406415 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:31.518333 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43655
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:31.596695 15613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:31.596894 15613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:31.596916 15613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:31.598066 15613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:31.598106 15613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:31.598120 15613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:31.598132 15613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:31.598186 15613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:31.599653 15613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:31.599905 15613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:31.600133 15613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:31.605048 15619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:31.605093 15621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:31.605113 15618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:31.605126 15613 server_base.cc:1034] running on GCE node
I20260926 22:44:31.605490 15613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:31.605774 15613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:31.606935 15613 hybrid_clock.cc:648] HybridClock initialized: now 1790462671606926 us; error 35 us; skew 500 ppm
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:31.609229 15613 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:31.609679 15613 webserver.cc:460] Webserver started at http://127.14.184.254:39385/ using document root <none> and password file <none>
I20260926 22:44:31.609967 15613 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:31.610033 15613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:31.610180 15613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:31.611421 15613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "eb042c59e1eb48748bb80b26bd3c696a"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "ed6da900ca77e7138ac997db1012de9e"
server_key_iv: "0f5c2c3382611a8b4440827375ffbea7"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.611776 15613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "eb042c59e1eb48748bb80b26bd3c696a"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "ed6da900ca77e7138ac997db1012de9e"
server_key_iv: "0f5c2c3382611a8b4440827375ffbea7"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.613049 15613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:31.613729 15628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.613902 15613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:31.613998 15613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "eb042c59e1eb48748bb80b26bd3c696a"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "ed6da900ca77e7138ac997db1012de9e"
server_key_iv: "0f5c2c3382611a8b4440827375ffbea7"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.614081 15613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:31.628289 15613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:31.630272 15613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:31.630419 15613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:31.634900 15613 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43655
I20260926 22:44:31.634938 15680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43655 every 8 connection(s)
I20260926 22:44:31.635257 15613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:31.636009 15681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:31.638157 15681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a: Bootstrap starting.
I20260926 22:44:31.638780 15681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:31.639056 15681 log.cc:824] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:31.639700 15681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a: No bootstrap required, opened a new log
I20260926 22:44:31.641211 15681 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb042c59e1eb48748bb80b26bd3c696a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43655 } }
I20260926 22:44:31.641376 15681 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:31.641448 15681 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb042c59e1eb48748bb80b26bd3c696a, State: Initialized, Role: FOLLOWER
I20260926 22:44:31.641609 15681 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb042c59e1eb48748bb80b26bd3c696a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43655 } }
I20260926 22:44:31.641690 15681 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:31.641731 15681 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:31.641779 15681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:31.642263 15681 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb042c59e1eb48748bb80b26bd3c696a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43655 } }
I20260926 22:44:31.642385 15681 leader_election.cc:302] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb042c59e1eb48748bb80b26bd3c696a; no voters:
I20260926 22:44:31.642571 15681 leader_election.cc:288] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:31.642593 15684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:31.642625 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15613
I20260926 22:44:31.642726 15684 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [term 1 LEADER]: Becoming Leader. State: Replica: eb042c59e1eb48748bb80b26bd3c696a, State: Running, Role: LEADER
I20260926 22:44:31.642735 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:31.642814 15684 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb042c59e1eb48748bb80b26bd3c696a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43655 } }
I20260926 22:44:31.642851 15681 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:31.642938 15075 external_mini_cluster.cc:1428] Setting key c747832ae05dcd39a0e3bdf13a38f4b4
I20260926 22:44:31.643133 15684 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb042c59e1eb48748bb80b26bd3c696a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb042c59e1eb48748bb80b26bd3c696a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43655 } } }
I20260926 22:44:31.643203 15684 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:31.643323 15684 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb042c59e1eb48748bb80b26bd3c696a. Latest consensus state: current_term: 1 leader_uuid: "eb042c59e1eb48748bb80b26bd3c696a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb042c59e1eb48748bb80b26bd3c696a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43655 } } }
I20260926 22:44:31.643388 15684 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:31.644032 15689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:31.644556 15689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:31.646123 15689 catalog_manager.cc:1347] Generated new cluster ID: 769a99f4324f4e378f99ff158081f98a
I20260926 22:44:31.646176 15689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:31.654250 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:31.643540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.14.184.254:43655)
0926 22:44:31.643737 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:31.643746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:31.643784 (+ 38us) server_negotiation.cc:407] Connection header received
0926 22:44:31.643973 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:31.643981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:31.644044 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:31.644199 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:31.644994 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.646234 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.647270 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.647849 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.647940 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:31.650461 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:31.650465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:31.650465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:31.650476 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:31.652395 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:31.652746 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:31.652748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:31.652749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:31.652896 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:31.653077 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:31.653078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:31.653078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:31.653355 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:31.653482 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:31.653911 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 22:44:31.654039 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260926 22:44:31.657222 15689 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:31.657698 15689 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:31.669447 15689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb042c59e1eb48748bb80b26bd3c696a: Generated new TSK 0
I20260926 22:44:31.669615 15689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:31.701495 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43655
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:31.781350 15707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:31.781544 15707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:31.781569 15707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:31.782637 15707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:31.782688 15707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:31.782757 15707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:31.784214 15707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:31.784389 15707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:31.784554 15707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:31.789604 15715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:31.789652 15707 server_base.cc:1034] running on GCE node
W20260926 22:44:31.789642 15713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:31.789605 15712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:31.790053 15707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:31.790306 15707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:31.791447 15707 hybrid_clock.cc:648] HybridClock initialized: now 1790462671791435 us; error 30 us; skew 500 ppm
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:31.793864 15707 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:31.794247 15707 webserver.cc:460] Webserver started at http://127.14.184.193:43385/ using document root <none> and password file <none>
I20260926 22:44:31.794450 15707 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:31.794492 15707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:31.794574 15707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:31.795641 15707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "664746e3ea5b41d7ac2446a731850a4b"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "ad40e31af97ff11649fba9973f770c3c"
server_key_iv: "56018ac9979f3c5e9f306022d34cd718"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.795928 15707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "664746e3ea5b41d7ac2446a731850a4b"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "ad40e31af97ff11649fba9973f770c3c"
server_key_iv: "56018ac9979f3c5e9f306022d34cd718"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.797140 15707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:31.797797 15722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.797963 15707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:31.798066 15707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "664746e3ea5b41d7ac2446a731850a4b"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "ad40e31af97ff11649fba9973f770c3c"
server_key_iv: "56018ac9979f3c5e9f306022d34cd718"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.798146 15707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:31.813813 15707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:31.815779 15707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:31.815951 15707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:31.816193 15707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:31.816529 15707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:31.816637 15707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.816701 15707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:31.816730 15707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.822481 15707 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:34741
I20260926 22:44:31.822561 15835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:34741 every 8 connection(s)
I20260926 22:44:31.822901 15707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:31.826143 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15707
I20260926 22:44:31.826251 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:31.826407 15075 external_mini_cluster.cc:1428] Setting key 876ac930d355db3c63d183bd155d2616
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:31.833050 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:31.823640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:49155 (local address 127.14.184.254:43655)
0926 22:44:31.823711 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:31.823713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:31.823940 (+ 227us) server_negotiation.cc:407] Connection header received
0926 22:44:31.824834 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:31.824835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:31.824847 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:31.824905 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:31.825474 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.826011 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.827376 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.827717 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.827792 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:31.830179 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:31.830184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:31.830185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:31.830191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:31.831657 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:31.832077 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:31.832079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:31.832079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:31.832141 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:31.832409 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:31.832409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:31.832410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:31.832520 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:31.832595 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:31.832870 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 22:44:31.832957 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 22:44:31.833256 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:31.823615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43655 (local address 127.14.184.193:49155)
0926 22:44:31.823794 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:31.823844 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:31.824737 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:31.824961 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:31.824964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:31.825211 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:31.825398 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:31.825401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.826098 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.826099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:31.827297 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:31.827301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.827813 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.827814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:31.827923 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:31.828448 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:31.828458 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:31.830071 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:31.831758 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:31.831766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:31.831768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:31.831997 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:31.832210 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:31.832211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:31.832211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:31.832333 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:31.832586 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:31.832593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:31.832736 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 22:44:31.832848 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 22:44:31.833003 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260926 22:44:31.833803 15836 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43655
I20260926 22:44:31.833890 15836 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:31.834074 15836 heartbeater.cc:511] Master 127.14.184.254:43655 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
I20260926 22:44:31.834576 15645 ts_manager.cc:194] Registered new tserver with Master: 664746e3ea5b41d7ac2446a731850a4b (127.14.184.193:34741)
I20260926 22:44:31.835199 15645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:49155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:31.851071 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43655
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:31.929667 15841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:31.929847 15841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:31.929870 15841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:31.931035 15841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:31.931069 15841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:31.931094 15841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:31.932524 15841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:31.932694 15841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:31.932857 15841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:31.937675 15841 server_base.cc:1034] running on GCE node
W20260926 22:44:31.937641 15847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:31.937656 15849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:31.937650 15846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:31.938090 15841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:31.938323 15841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:31.939456 15841 hybrid_clock.cc:648] HybridClock initialized: now 1790462671939445 us; error 28 us; skew 500 ppm
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:31.941955 15841 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:31.942327 15841 webserver.cc:460] Webserver started at http://127.14.184.194:34629/ using document root <none> and password file <none>
I20260926 22:44:31.942529 15841 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:31.942569 15841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:31.942655 15841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:31.943802 15841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "dae2a0bbf79c40808a8e1c2935e6e755"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "8a378ef4fe189553a6ccf0a44b6dc698"
server_key_iv: "d27d4fcd547096a7c1f3a73951b1c56d"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.944139 15841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "dae2a0bbf79c40808a8e1c2935e6e755"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "8a378ef4fe189553a6ccf0a44b6dc698"
server_key_iv: "d27d4fcd547096a7c1f3a73951b1c56d"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.945392 15841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:31.946012 15856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.946162 15841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:31.946240 15841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "dae2a0bbf79c40808a8e1c2935e6e755"
format_stamp: "Formatted at 2026-09-26 22:44:31 on dist-test-slave-3986"
server_key: "8a378ef4fe189553a6ccf0a44b6dc698"
server_key_iv: "d27d4fcd547096a7c1f3a73951b1c56d"
server_key_version: "encryptionkey@0"
I20260926 22:44:31.946291 15841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:31.954578 15841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:31.956313 15841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:31.956418 15841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:31.956576 15841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:31.956805 15841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:31.956833 15841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.956853 15841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:31.956866 15841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:31.962284 15841 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:37111
I20260926 22:44:31.962363 15969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:37111 every 8 connection(s)
I20260926 22:44:31.962790 15841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:31.964987 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15841
I20260926 22:44:31.965057 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:31.965226 15075 external_mini_cluster.cc:1428] Setting key a01da4ded432bf798ce6da8e6147ecb2
Sep 26 22:44:31 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:31.972535 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:31.963464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:37015 (local address 127.14.184.254:43655)
0926 22:44:31.963508 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:31.963510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:31.963718 (+ 208us) server_negotiation.cc:407] Connection header received
0926 22:44:31.964620 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:31.964621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:31.964634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:31.964683 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:31.965374 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.966014 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.967098 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.967323 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.967408 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:31.969790 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:31.969791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:31.969791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:31.969805 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:31.971164 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:31.971630 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:31.971631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:31.971631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:31.971677 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:31.971945 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:31.971945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:31.971946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:31.972037 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:31.972086 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:31.972417 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 22:44:31.972474 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:31.972855 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:31.963424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43655 (local address 127.14.184.194:37015)
0926 22:44:31.963599 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:31.963649 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:31.964510 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:31.964762 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:31.964766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:31.965026 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:31.965279 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:31.965285 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.966101 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.966102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:31.967002 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:31.967004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:31.967418 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:31.967419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:31.967544 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:31.968047 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:31.968054 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:31.969701 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:31.971248 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:31.971255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:31.971257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:31.971554 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:31.971741 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:31.971742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:31.971743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:31.971877 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:31.972100 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:31.972106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:31.972260 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 22:44:31.972388 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 22:44:31.972544 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260926 22:44:31.973377 15970 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43655
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
I20260926 22:44:31.973479 15970 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:31.973664 15970 heartbeater.cc:511] Master 127.14.184.254:43655 requested a full tablet report, sending...
I20260926 22:44:31.974026 15645 ts_manager.cc:194] Registered new tserver with Master: dae2a0bbf79c40808a8e1c2935e6e755 (127.14.184.194:37111)
I20260926 22:44:31.974397 15645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:37015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:31.989567 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43655
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:32.068391 15975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:32.068619 15975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:32.068658 15975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:32.069811 15975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:32.069877 15975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:32.069923 15975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:32.071439 15975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:32.071702 15975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:32.071920 15975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:32.076678 15980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:32.076713 15981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:32.076861 15975 server_base.cc:1034] running on GCE node
W20260926 22:44:32.076695 15983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:32.077153 15975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:32.077397 15975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:32.078569 15975 hybrid_clock.cc:648] HybridClock initialized: now 1790462672078567 us; error 35 us; skew 500 ppm
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462672, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:32.080972 15975 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:32.081401 15975 webserver.cc:460] Webserver started at http://127.14.184.195:46253/ using document root <none> and password file <none>
I20260926 22:44:32.081624 15975 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:32.081669 15975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:32.081831 15975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:32.082921 15975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "2a4aed5d2db845e5bb06ac4649845e2d"
format_stamp: "Formatted at 2026-09-26 22:44:32 on dist-test-slave-3986"
server_key: "dec500ccb2ec4c07319b1a9de2096782"
server_key_iv: "02fdfebb0658e90bba1319d12a066d73"
server_key_version: "encryptionkey@0"
I20260926 22:44:32.083266 15975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "2a4aed5d2db845e5bb06ac4649845e2d"
format_stamp: "Formatted at 2026-09-26 22:44:32 on dist-test-slave-3986"
server_key: "dec500ccb2ec4c07319b1a9de2096782"
server_key_iv: "02fdfebb0658e90bba1319d12a066d73"
server_key_version: "encryptionkey@0"
I20260926 22:44:32.084599 15975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:32.085264 15990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:32.085438 15975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:32.085536 15975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "2a4aed5d2db845e5bb06ac4649845e2d"
format_stamp: "Formatted at 2026-09-26 22:44:32 on dist-test-slave-3986"
server_key: "dec500ccb2ec4c07319b1a9de2096782"
server_key_iv: "02fdfebb0658e90bba1319d12a066d73"
server_key_version: "encryptionkey@0"
I20260926 22:44:32.085613 15975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:32.104718 15975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:32.106657 15975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:32.106849 15975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:32.107070 15975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:32.107379 15975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:32.107419 15975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:32.107460 15975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:32.107491 15975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:32.113108 15975 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33767
I20260926 22:44:32.113161 16103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33767 every 8 connection(s)
I20260926 22:44:32.113497 15975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:32.114032 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 15975
I20260926 22:44:32.114144 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:32.114348 15075 external_mini_cluster.cc:1428] Setting key f4ef2ae698c6662d1bb130b7c8234da8
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462672, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:32.122856 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.114191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:39633 (local address 127.14.184.254:43655)
0926 22:44:32.114248 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.114250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.114515 (+ 265us) server_negotiation.cc:407] Connection header received
0926 22:44:32.115663 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.115665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.115678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.115728 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:32.116363 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.116868 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.117954 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.118160 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.118245 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.120257 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:32.120258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:32.120258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:32.120264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:32.121602 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.121991 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.121992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.121992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.122041 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.122243 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.122244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.122244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.122349 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:32.122399 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.122623 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.122739 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:44:32.123155 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.114179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43655 (local address 127.14.184.195:39633)
0926 22:44:32.114378 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:32.114430 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:32.115553 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:32.115793 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:32.115797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:32.116050 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:32.116260 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.116266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.116993 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.116994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.117849 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.117852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.118254 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.118255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.118379 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.118838 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:32.118846 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:32.120173 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:32.121706 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:32.121712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:32.121714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:32.121928 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:32.122098 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:32.122099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:32.122099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:32.122190 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:32.122404 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:32.122408 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:32.122514 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 22:44:32.122595 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 22:44:32.122771 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260926 22:44:32.123674 16104 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43655
I20260926 22:44:32.123754 16104 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:32.123888 16104 heartbeater.cc:511] Master 127.14.184.254:43655 requested a full tablet report, sending...
I20260926 22:44:32.124217 15645 ts_manager.cc:194] Registered new tserver with Master: 2a4aed5d2db845e5bb06ac4649845e2d (127.14.184.195:33767)
I20260926 22:44:32.124595 15645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:39633
I20260926 22:44:32.125948 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:44:32.132921 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.127543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.14.184.254:43655)
0926 22:44:32.127601 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.127603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.127677 (+ 74us) server_negotiation.cc:407] Connection header received
0926 22:44:32.127728 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.127729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.127737 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.127784 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:32.128322 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.128762 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.129432 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.129604 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.129652 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.130855 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:32.130856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:32.130856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:32.130861 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:32.131995 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.132303 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.132304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.132304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.132351 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.132511 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.132512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.132512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.132638 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:32.132715 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.132804 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.132860 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:44:32.135192 15645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:32.136737 15645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:32.143151 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.138771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33767 (local address 127.0.0.1:36072)
0926 22:44:32.139024 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:32.139035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:32.139083 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:32.139552 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:32.139552 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:32.139555 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:32.139591 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.139592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.140852 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.140853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.141889 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.141892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.142840 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.142841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.142924 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.142937 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 22:44:32.143009 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 22:44:32.143066 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":71,"thread_start_us":85,"threads_started":1}
I20260926 22:44:32.143384 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.138770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:34741 (local address 127.0.0.1:49526)
0926 22:44:32.138919 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:32.138957 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:32.139029 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:32.139469 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:32.139472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:32.139474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:32.139550 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.139552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.140943 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.140944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.142083 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.142085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.143104 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.143105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.143155 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.143158 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:32.143210 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:44:32.143258 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":26,"thread_start_us":55,"threads_started":1}
I20260926 22:44:32.143427 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.138952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.14.184.195:33767)
0926 22:44:32.139382 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.139388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.139398 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:32.139440 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.139442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.139458 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.139543 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:32.139681 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.140731 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.141979 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.142735 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.142808 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.142863 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.143306 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.143349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":95,"threads_started":1}
I20260926 22:44:32.143397 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.138988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:37111 (local address 127.0.0.1:45240)
0926 22:44:32.139195 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:32.139211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:32.139245 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:32.139843 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:32.139844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:32.139847 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:32.139889 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.139891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.140728 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.140728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.142082 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.142085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.143061 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.143062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.143139 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.143144 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:44:32.143289 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 22:44:32.143328 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":45,"thread_start_us":73,"threads_started":1}
I20260926 22:44:32.143762 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.139224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.14.184.194:37111)
0926 22:44:32.139408 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.139412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.139421 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:32.139456 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.139458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.139471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.139751 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:32.139968 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.140615 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.142169 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.142886 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.142978 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.143031 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.143644 (+ 613us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.143693 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260926 22:44:32.143900 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.138863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49526 (local address 127.14.184.193:34741)
0926 22:44:32.139197 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.139203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.139214 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:32.139255 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.139257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.139276 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.139370 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:32.139636 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.140756 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.142234 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.143037 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.143104 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.143156 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.143221 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.143789 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":76,"threads_started":1}
I20260926 22:44:32.144172 16038 tablet_service.cc:1467] Processing CreateTablet for tablet 0b814ad1d9734095acb9b861bd2305b7 (DEFAULT_TABLE table=test-table [id=286d0ce81da4448b9f8e9a3e4729f63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:32.144567 16038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b814ad1d9734095acb9b861bd2305b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:32.144562 15770 tablet_service.cc:1467] Processing CreateTablet for tablet 0b814ad1d9734095acb9b861bd2305b7 (DEFAULT_TABLE table=test-table [id=286d0ce81da4448b9f8e9a3e4729f63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:32.144898 15770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b814ad1d9734095acb9b861bd2305b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:32.148108 16119 tablet_bootstrap.cc:492] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: Bootstrap starting.
I20260926 22:44:32.148106 16118 tablet_bootstrap.cc:492] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: Bootstrap starting.
I20260926 22:44:32.148366 15904 tablet_service.cc:1467] Processing CreateTablet for tablet 0b814ad1d9734095acb9b861bd2305b7 (DEFAULT_TABLE table=test-table [id=286d0ce81da4448b9f8e9a3e4729f63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:32.148661 15904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b814ad1d9734095acb9b861bd2305b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:32.148857 16118 tablet_bootstrap.cc:654] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:32.148857 16119 tablet_bootstrap.cc:654] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:32.149513 16119 log.cc:824] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:32.149513 16118 log.cc:824] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:32.150359 16118 tablet_bootstrap.cc:492] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: No bootstrap required, opened a new log
I20260926 22:44:32.150429 16118 ts_tablet_manager.cc:1397] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:32.150494 16119 tablet_bootstrap.cc:492] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: No bootstrap required, opened a new log
I20260926 22:44:32.150554 16119 ts_tablet_manager.cc:1397] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 22:44:32.151592 16122 tablet_bootstrap.cc:492] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: Bootstrap starting.
I20260926 22:44:32.152232 16122 tablet_bootstrap.cc:654] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:32.152447 16118 raft_consensus.cc:348] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.152493 16119 raft_consensus.cc:348] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.152637 16119 raft_consensus.cc:374] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:32.152637 16118 raft_consensus.cc:374] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:32.152671 16119 raft_consensus.cc:729] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664746e3ea5b41d7ac2446a731850a4b, State: Initialized, Role: FOLLOWER
I20260926 22:44:32.152671 16118 raft_consensus.cc:729] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4aed5d2db845e5bb06ac4649845e2d, State: Initialized, Role: FOLLOWER
I20260926 22:44:32.152776 16122 log.cc:824] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:32.152812 16118 consensus_queue.cc:260] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.152817 16119 consensus_queue.cc:260] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.153079 16119 ts_tablet_manager.cc:1428] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:44:32.153079 16118 ts_tablet_manager.cc:1428] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:44:32.153198 16104 heartbeater.cc:503] Master 127.14.184.254:43655 was elected leader, sending a full tablet report...
I20260926 22:44:32.153198 15836 heartbeater.cc:503] Master 127.14.184.254:43655 was elected leader, sending a full tablet report...
I20260926 22:44:32.153558 16122 tablet_bootstrap.cc:492] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: No bootstrap required, opened a new log
I20260926 22:44:32.153628 16122 ts_tablet_manager.cc:1397] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:32.155416 16122 raft_consensus.cc:348] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.155543 16122 raft_consensus.cc:374] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:32.155565 16122 raft_consensus.cc:729] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dae2a0bbf79c40808a8e1c2935e6e755, State: Initialized, Role: FOLLOWER
I20260926 22:44:32.155701 16122 consensus_queue.cc:260] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.155995 15970 heartbeater.cc:503] Master 127.14.184.254:43655 was elected leader, sending a full tablet report...
I20260926 22:44:32.155995 16122 ts_tablet_manager.cc:1428] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260926 22:44:32.213361 15971 tablet.cc:2369] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:44:32.323664 15837 tablet.cc:2369] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:44:32.364113 16105 tablet.cc:2369] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:32.481931 16126 raft_consensus.cc:482] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:32.482051 16126 raft_consensus.cc:504] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.482353 16126 leader_election.cc:288] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 664746e3ea5b41d7ac2446a731850a4b (127.14.184.193:34741), 2a4aed5d2db845e5bb06ac4649845e2d (127.14.184.195:33767)
I20260926 22:44:32.486363 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.482620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:52605 (local address 127.14.184.195:33767)
0926 22:44:32.482665 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.482668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.482779 (+ 111us) server_negotiation.cc:407] Connection header received
0926 22:44:32.482868 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.482876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.482887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.482945 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:32.483182 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.483934 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.485449 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.486005 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.486069 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.486077 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.486243 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.486281 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:32.486363 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.482621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:54423 (local address 127.14.184.193:34741)
0926 22:44:32.482665 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.482668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.482779 (+ 111us) server_negotiation.cc:407] Connection header received
0926 22:44:32.482877 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.482878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.482887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.482945 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:32.483182 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.483924 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.485448 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.485991 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.486068 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.486077 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.486243 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.486280 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:32.486418 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.482535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:34741 (local address 127.14.184.194:54423)
0926 22:44:32.482698 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:32.482707 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:32.482772 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:32.482966 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:32.482967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:32.482970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:32.483097 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.483099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.484009 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.484010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.485359 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.485362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.486080 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.486081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.486162 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.486169 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 22:44:32.486227 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 22:44:32.486280 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":94,"thread_start_us":78,"threads_started":1}
I20260926 22:44:32.486420 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.482533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33767 (local address 127.14.184.194:52605)
0926 22:44:32.482674 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:32.482702 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:32.482772 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:32.482966 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:32.482967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:32.482970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:32.483097 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.483099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.484010 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.484010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.485359 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:32.485362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.486079 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.486081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:32.486162 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.486170 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 22:44:32.486227 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 22:44:32.486280 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":57,"thread_start_us":79,"threads_started":1}
I20260926 22:44:32.486788 15790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b814ad1d9734095acb9b861bd2305b7" candidate_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "664746e3ea5b41d7ac2446a731850a4b" is_pre_election: true
I20260926 22:44:32.486788 16058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b814ad1d9734095acb9b861bd2305b7" candidate_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" is_pre_election: true
I20260926 22:44:32.486936 16058 raft_consensus.cc:2454] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dae2a0bbf79c40808a8e1c2935e6e755 in term 0.
I20260926 22:44:32.486936 15790 raft_consensus.cc:2454] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dae2a0bbf79c40808a8e1c2935e6e755 in term 0.
I20260926 22:44:32.487144 15857 leader_election.cc:302] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 664746e3ea5b41d7ac2446a731850a4b, dae2a0bbf79c40808a8e1c2935e6e755; no voters:
I20260926 22:44:32.487308 16126 raft_consensus.cc:2781] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:32.487375 16126 raft_consensus.cc:482] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:32.487397 16126 raft_consensus.cc:3037] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:32.488036 16126 raft_consensus.cc:504] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.488163 16126 leader_election.cc:288] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [CANDIDATE]: Term 1 election: Requested vote from peers 664746e3ea5b41d7ac2446a731850a4b (127.14.184.193:34741), 2a4aed5d2db845e5bb06ac4649845e2d (127.14.184.195:33767)
I20260926 22:44:32.488356 15790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b814ad1d9734095acb9b861bd2305b7" candidate_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "664746e3ea5b41d7ac2446a731850a4b"
I20260926 22:44:32.488376 16058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b814ad1d9734095acb9b861bd2305b7" candidate_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4aed5d2db845e5bb06ac4649845e2d"
I20260926 22:44:32.488448 15790 raft_consensus.cc:3037] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:32.488445 16058 raft_consensus.cc:3037] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:32.489115 15790 raft_consensus.cc:2454] T 0b814ad1d9734095acb9b861bd2305b7 P 664746e3ea5b41d7ac2446a731850a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dae2a0bbf79c40808a8e1c2935e6e755 in term 1.
I20260926 22:44:32.489277 16058 raft_consensus.cc:2454] T 0b814ad1d9734095acb9b861bd2305b7 P 2a4aed5d2db845e5bb06ac4649845e2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dae2a0bbf79c40808a8e1c2935e6e755 in term 1.
I20260926 22:44:32.489305 15857 leader_election.cc:302] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 664746e3ea5b41d7ac2446a731850a4b, dae2a0bbf79c40808a8e1c2935e6e755; no voters:
I20260926 22:44:32.489419 16126 raft_consensus.cc:2781] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:32.489563 16126 raft_consensus.cc:686] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [term 1 LEADER]: Becoming Leader. State: Replica: dae2a0bbf79c40808a8e1c2935e6e755, State: Running, Role: LEADER
I20260926 22:44:32.489670 16126 consensus_queue.cc:237] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } }
I20260926 22:44:32.490479 15645 catalog_manager.cc:5495] T 0b814ad1d9734095acb9b861bd2305b7 P dae2a0bbf79c40808a8e1c2935e6e755 reported cstate change: term changed from 0 to 1, leader changed from <none> to dae2a0bbf79c40808a8e1c2935e6e755 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "664746e3ea5b41d7ac2446a731850a4b" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4aed5d2db845e5bb06ac4649845e2d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dae2a0bbf79c40808a8e1c2935e6e755" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37111 } health_report { overall_health: HEALTHY } } }
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:32.505426 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.498406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.14.184.193:34741)
0926 22:44:32.498445 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.498447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.498564 (+ 117us) server_negotiation.cc:407] Connection header received
0926 22:44:32.498641 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.498642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.498651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.498723 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:32.499247 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.499698 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.500469 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.500678 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.500750 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.503074 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:32.503083 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:32.503084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:32.503090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:32.504450 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.504761 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.504764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.504764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.504826 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.504993 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.504993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.504993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.505112 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:32.505221 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.505268 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.505333 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462672, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462672, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:32.531945 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.521238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.14.184.193:34741)
0926 22:44:32.521313 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.521316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.521394 (+ 78us) server_negotiation.cc:407] Connection header received
0926 22:44:32.521472 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.521473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.521487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.521543 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:32.522060 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.522521 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.523316 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.523509 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.523591 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.529616 (+ 6025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:32.529617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:32.529618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:32.529625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:32.530779 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.531128 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.531129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.531129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.531182 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.531379 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.531380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.531380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.531477 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:32.531571 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.531760 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.531843 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[15600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:32.553146 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:32.546606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.14.184.193:34741)
0926 22:44:32.546716 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:32.546718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:32.546814 (+ 96us) server_negotiation.cc:407] Connection header received
0926 22:44:32.546894 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:32.546895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:32.546905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:32.546971 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:32.547493 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.547945 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.548727 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:32.548909 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:32.548981 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:32.551061 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:32.551063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:32.551063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:32.551070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:32.552126 (+ 1056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.552441 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.552442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.552443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.552496 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:32.552687 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:32.552687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:32.552688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:32.552778 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:32.552873 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:32.552978 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 22:44:32.553048 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 22:44:32.553735 15750 tablet_service.cc:2908] Rejecting scan request for tablet 0b814ad1d9734095acb9b861bd2305b7: Uninitialized: clean time has not yet been initialized
I20260926 22:44:32.554826 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15707
I20260926 22:44:32.559468 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15841
I20260926 22:44:32.563993 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15975
I20260926 22:44:32.568269 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 15613
2026-09-26T22:44:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2198 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[16143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[16143](info): set up 2 sockets
Sep 26 22:44:32 dist-test-slave-3986 krb5kdc[16143](info): commencing operation
krb5kdc: starting...
W20260926 22:44:33.602440 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:33 dist-test-slave-3986 krb5kdc[16143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:33.715372 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:40689
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:40689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:33.794296 16156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:33.794534 16156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:33.794570 16156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:33.795753 16156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:33.795831 16156 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 22:44:33.795857 16156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:33.795888 16156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:33.795917 16156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:33.795964 16156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:33.797410 16156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:40689
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:40689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:33.797673 16156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:33.797890 16156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:33.802775 16156 server_base.cc:1034] running on GCE node
W20260926 22:44:33.802693 16162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:33.802697 16161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:33.802724 16164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:33.803300 16156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:33.803604 16156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:33.804770 16156 hybrid_clock.cc:648] HybridClock initialized: now 1790462673804757 us; error 45 us; skew 500 ppm
Sep 26 22:44:33 dist-test-slave-3986 krb5kdc[16143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462673, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:33.811443 16156 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:33.811825 16156 webserver.cc:460] Webserver started at http://127.14.184.254:40607/ using document root <none> and password file <none>
I20260926 22:44:33.812018 16156 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:33.812057 16156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:33.812144 16156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:33.813521 16156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "ca7f3520cf5746ec8b5491ad727d8340"
format_stamp: "Formatted at 2026-09-26 22:44:33 on dist-test-slave-3986"
server_key: "4d879ef5e39bd3722a85fb1870ba077d"
server_key_iv: "a1148a53f8e06ce6090041f3b0950f72"
server_key_version: "encryptionkey@0"
I20260926 22:44:33.813946 16156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "ca7f3520cf5746ec8b5491ad727d8340"
format_stamp: "Formatted at 2026-09-26 22:44:33 on dist-test-slave-3986"
server_key: "4d879ef5e39bd3722a85fb1870ba077d"
server_key_iv: "a1148a53f8e06ce6090041f3b0950f72"
server_key_version: "encryptionkey@0"
I20260926 22:44:33.815339 16156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:33.816018 16171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:33.816226 16156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:33.816305 16156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "ca7f3520cf5746ec8b5491ad727d8340"
format_stamp: "Formatted at 2026-09-26 22:44:33 on dist-test-slave-3986"
server_key: "4d879ef5e39bd3722a85fb1870ba077d"
server_key_iv: "a1148a53f8e06ce6090041f3b0950f72"
server_key_version: "encryptionkey@0"
I20260926 22:44:33.816355 16156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:33.828168 16156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:33.829864 16156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:33.829968 16156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:33.834470 16156 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:40689
I20260926 22:44:33.834560 16233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:40689 every 8 connection(s)
I20260926 22:44:33.834890 16156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:33.835569 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:33.837826 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: Bootstrap starting.
I20260926 22:44:33.838402 16234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:33.838713 16234 log.cc:824] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:33.839383 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: No bootstrap required, opened a new log
I20260926 22:44:33.839835 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 16156
I20260926 22:44:33.839913 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:33.840062 15075 external_mini_cluster.cc:1428] Setting key 67adb4dfc9b1f95800afd1325a902d57
I20260926 22:44:33.842028 16234 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f3520cf5746ec8b5491ad727d8340" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40689 } }
I20260926 22:44:33.842198 16234 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:33.842235 16234 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7f3520cf5746ec8b5491ad727d8340, State: Initialized, Role: FOLLOWER
I20260926 22:44:33.842397 16234 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f3520cf5746ec8b5491ad727d8340" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40689 } }
I20260926 22:44:33.842461 16234 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:33.842481 16234 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:33.842545 16234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:33.843178 16234 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f3520cf5746ec8b5491ad727d8340" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40689 } }
I20260926 22:44:33.843317 16234 leader_election.cc:302] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca7f3520cf5746ec8b5491ad727d8340; no voters:
I20260926 22:44:33.843495 16234 leader_election.cc:288] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:33.843555 16239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:33.843724 16234 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:33.843724 16239 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [term 1 LEADER]: Becoming Leader. State: Replica: ca7f3520cf5746ec8b5491ad727d8340, State: Running, Role: LEADER
I20260926 22:44:33.843974 16239 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f3520cf5746ec8b5491ad727d8340" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40689 } }
I20260926 22:44:33.844367 16240 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca7f3520cf5746ec8b5491ad727d8340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f3520cf5746ec8b5491ad727d8340" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40689 } } }
I20260926 22:44:33.844450 16240 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:33.844658 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca7f3520cf5746ec8b5491ad727d8340. Latest consensus state: current_term: 1 leader_uuid: "ca7f3520cf5746ec8b5491ad727d8340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7f3520cf5746ec8b5491ad727d8340" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40689 } } }
I20260926 22:44:33.844962 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340 [sys.catalog]: This master's current role is: LEADER
W20260926 22:44:33.845510 16255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:44:33.845570 16255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:44:33.845650 16248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:33.845827 16248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:33.847267 16248 catalog_manager.cc:1347] Generated new cluster ID: 5d1e1057a04a43099e7653d99a2fdd2e
I20260926 22:44:33.847322 16248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:33 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:33.852631 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:33.840565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.14.184.254:40689)
0926 22:44:33.840756 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:33.840766 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:33.840805 (+ 39us) server_negotiation.cc:407] Connection header received
0926 22:44:33.841994 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:33.842002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:33.842064 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:33.842208 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:33.843023 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:33.844452 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:33.845623 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:33.846214 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:33.846347 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:33.848778 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:33.848783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:33.848783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:33.848794 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:33.851089 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:33.851478 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:33.851481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:33.851481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:33.851590 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:33.851816 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:33.851817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:33.851818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:33.852042 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:33.852224 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:33.852305 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 22:44:33.852445 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260926 22:44:33.872058 16248 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:33.872588 16248 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:33.877077 16248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: Generated new TSK 0
I20260926 22:44:33.877261 16248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:33.907027 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40689
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:33.997848 16260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:33.998028 16260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:33.998052 16260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:33.999191 16260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:33.999233 16260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:33.999253 16260 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:33.999269 16260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:34.000584 16260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:34.000774 16260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:34.000937 16260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:34.005651 16266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:34.005657 16268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:34.005679 16265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:34.005874 16260 server_base.cc:1034] running on GCE node
I20260926 22:44:34.006110 16260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:34.006383 16260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:34.007540 16260 hybrid_clock.cc:648] HybridClock initialized: now 1790462674007535 us; error 36 us; skew 500 ppm
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:34.010046 16260 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:34.010419 16260 webserver.cc:460] Webserver started at http://127.14.184.193:45193/ using document root <none> and password file <none>
I20260926 22:44:34.010605 16260 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:34.010644 16260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:34.010814 16260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:34.011785 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "6e6378e119b24965b48f47d136285620"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "517968d7339f351657e8675ea1b543b9"
server_key_iv: "5dcf1a2db620873f6b32a049ade0ad26"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.012069 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "6e6378e119b24965b48f47d136285620"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "517968d7339f351657e8675ea1b543b9"
server_key_iv: "5dcf1a2db620873f6b32a049ade0ad26"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.013247 16260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:34.013887 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.014055 16260 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:34.014179 16260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "6e6378e119b24965b48f47d136285620"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "517968d7339f351657e8675ea1b543b9"
server_key_iv: "5dcf1a2db620873f6b32a049ade0ad26"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.014252 16260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:34.028671 16260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:34.030422 16260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:34.030573 16260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:34.031073 16260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:34.031137 16260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.031184 16260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:34.031212 16260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.037485 16260 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:43037
I20260926 22:44:34.037600 16390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:43037 every 8 connection(s)
I20260926 22:44:34.037918 16260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:34.041206 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.031524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:38375 (local address 127.14.184.254:40689)
0926 22:44:34.031603 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.031605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.031793 (+ 188us) server_negotiation.cc:407] Connection header received
0926 22:44:34.032284 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.032286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.032298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.032360 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.033199 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.034059 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.035660 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.036015 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.036108 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.038632 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.038634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.038634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.038641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.039855 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.040239 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.040240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.040241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.040300 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.040548 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.040548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.040549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.040667 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.040748 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.041016 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.041115 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 22:44:34.041342 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.031509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.14.184.193:38375)
0926 22:44:34.031664 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.031708 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.032175 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.032431 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.032436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.032814 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.033086 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.033092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.034180 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.034181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.035558 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.035562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.036107 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.036108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.036208 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.036761 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.036778 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.038535 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.039941 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.039947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.039950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.040161 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.040372 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.040373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.040373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.040481 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.040740 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.040746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.040878 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.040980 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.041125 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
I20260926 22:44:34.041842 16391 heartbeater.cc:347] Connected to a master server at 127.14.184.254:40689
I20260926 22:44:34.041944 16391 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:34.042023 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 16260
I20260926 22:44:34.042124 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:34.042140 16391 heartbeater.cc:511] Master 127.14.184.254:40689 requested a full tablet report, sending...
I20260926 22:44:34.042318 15075 external_mini_cluster.cc:1428] Setting key 7b5342fd19b51f3c7dc24d748b9f6993
I20260926 22:44:34.042711 16187 ts_manager.cc:194] Registered new tserver with Master: 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037)
I20260926 22:44:34.043535 16187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:38375
I20260926 22:44:34.048859 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.042907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.14.184.193:40755)
0926 22:44:34.043054 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.043066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.043129 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.043250 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.043251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.043451 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.043522 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.043524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.044215 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.044216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.045000 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.045003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.045399 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.045400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.045475 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.045892 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.045897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.046175 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.047964 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.047965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.047965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.048117 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.048325 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.048325 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.048326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.048372 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.048614 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.048615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.048655 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.048715 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.048794 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":8,"thread_start_us":77,"threads_started":1}
I20260926 22:44:34.048959 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.042869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40755 (local address 127.14.184.254:40689)
0926 22:44:34.042988 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.042990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.043136 (+ 146us) server_negotiation.cc:407] Connection header received
0926 22:44:34.043183 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.043183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.043192 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.043247 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.043620 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.044115 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.045083 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.045321 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.045388 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.046255 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.046257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.046258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.046264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.047885 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.048190 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.048191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.048192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.048254 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.048415 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.048416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.048416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.048552 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.048619 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.048764 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.048845 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:34.067102 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40689
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:34.146634 16400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:34.146885 16400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:34.146926 16400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:34.148066 16400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:34.148133 16400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:34.148168 16400 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:34.148207 16400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:34.149608 16400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:34.149885 16400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:34.150105 16400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:34.154883 16405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:34.154858 16408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:34.155088 16400 server_base.cc:1034] running on GCE node
W20260926 22:44:34.154867 16406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:34.155346 16400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:34.155601 16400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:34.156733 16400 hybrid_clock.cc:648] HybridClock initialized: now 1790462674156716 us; error 40 us; skew 500 ppm
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:34.159126 16400 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:34.159575 16400 webserver.cc:460] Webserver started at http://127.14.184.194:35021/ using document root <none> and password file <none>
I20260926 22:44:34.159749 16400 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:34.159788 16400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:34.159873 16400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:34.160796 16400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "4352922687774180acaf347ec6631734"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "c34c69761cd85d7e00b270f6a33e11b4"
server_key_iv: "7ad6d7a87c54d84719f56fbf90678110"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.161141 16400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "4352922687774180acaf347ec6631734"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "c34c69761cd85d7e00b270f6a33e11b4"
server_key_iv: "7ad6d7a87c54d84719f56fbf90678110"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.162415 16400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:34.163200 16415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.163388 16400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.163473 16400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "4352922687774180acaf347ec6631734"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "c34c69761cd85d7e00b270f6a33e11b4"
server_key_iv: "7ad6d7a87c54d84719f56fbf90678110"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.163544 16400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:34.171187 16400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:34.172896 16400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:34.173056 16400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:34.173614 16400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:34.173674 16400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.173708 16400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:34.173735 16400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:34.180897 16400 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:38845
I20260926 22:44:34.180979 16530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:38845 every 8 connection(s)
I20260926 22:44:34.181265 16400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:34.183604 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.173988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:37957 (local address 127.14.184.254:40689)
0926 22:44:34.174060 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.174063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.174243 (+ 180us) server_negotiation.cc:407] Connection header received
0926 22:44:34.174729 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.174730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.174746 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.174837 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.175465 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.176304 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.177995 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.178326 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.178427 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.180992 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.180993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.180994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.181001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.182464 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.182892 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.182893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.182893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.182943 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.183146 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.183146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.183146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.183235 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.183279 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.183488 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.183544 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 22:44:34.183766 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.173967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.14.184.194:37957)
0926 22:44:34.174104 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.174143 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.174599 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.174888 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.174891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.175179 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.175361 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.175366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.176410 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.176411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.177888 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.177893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.178456 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.178457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.178620 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.179147 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.179167 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.180861 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.182539 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.182542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.182544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.182828 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.183002 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.183003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.183003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.183098 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.183283 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.183287 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.183394 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.183469 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.183579 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260926 22:44:34.184248 16531 heartbeater.cc:347] Connected to a master server at 127.14.184.254:40689
I20260926 22:44:34.184353 16531 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:34.184521 16531 heartbeater.cc:511] Master 127.14.184.254:40689 requested a full tablet report, sending...
I20260926 22:44:34.184927 16187 ts_manager.cc:194] Registered new tserver with Master: 4352922687774180acaf347ec6631734 (127.14.184.194:38845)
I20260926 22:44:34.185410 16187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:37957
I20260926 22:44:34.190304 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.185319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:32911 (local address 127.14.184.254:40689)
0926 22:44:34.185362 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.185364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.185515 (+ 151us) server_negotiation.cc:407] Connection header received
0926 22:44:34.185561 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.185561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.185571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.185624 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.185962 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.186488 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.187205 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.187388 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.187440 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.188150 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.188151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.188151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.188157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.189482 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.189748 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.189749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.189749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.189789 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.189921 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.189921 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.189921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.190036 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.190083 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.190190 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.190241 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:44:34.190315 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.185259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.14.184.194:32911)
0926 22:44:34.185413 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.185423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.185505 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.185626 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.185627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.185829 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.185885 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.185886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.186573 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.186574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.187139 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.187141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.187452 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.187453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.187504 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.187833 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.187837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.188080 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.189550 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.189551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.189551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.189695 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.189837 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.189838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.189838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.189879 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.190089 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.190090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.190125 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.190176 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.190246 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260926 22:44:34.191385 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 16400
I20260926 22:44:34.191488 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:34.191659 15075 external_mini_cluster.cc:1428] Setting key e966435c36f277542a985adc89143b9e
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:34.214918 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40689
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:34.293407 16540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:34.293617 16540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:34.293653 16540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:34.294798 16540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:34.294862 16540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:34.294896 16540 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:34.294937 16540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:34.296315 16540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:34.296559 16540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:34.296773 16540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:34.301481 16548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:34.301461 16546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:34.301563 16540 server_base.cc:1034] running on GCE node
W20260926 22:44:34.301460 16545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:34.301854 16540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:34.302095 16540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:34.303258 16540 hybrid_clock.cc:648] HybridClock initialized: now 1790462674303248 us; error 35 us; skew 500 ppm
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:34.305629 16540 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:34.306056 16540 webserver.cc:460] Webserver started at http://127.14.184.195:42917/ using document root <none> and password file <none>
I20260926 22:44:34.306298 16540 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:34.306365 16540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:34.306514 16540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:34.307581 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "6a8e8388263c4d53a47a2576dbf62e6d"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "0720632a128b0347889a439567f5d572"
server_key_iv: "1556367b1596d6ae80f1174ab7b96b64"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.307940 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "6a8e8388263c4d53a47a2576dbf62e6d"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "0720632a128b0347889a439567f5d572"
server_key_iv: "1556367b1596d6ae80f1174ab7b96b64"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.309271 16540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:34.309978 16555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.310143 16540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:34.310240 16540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "6a8e8388263c4d53a47a2576dbf62e6d"
format_stamp: "Formatted at 2026-09-26 22:44:34 on dist-test-slave-3986"
server_key: "0720632a128b0347889a439567f5d572"
server_key_iv: "1556367b1596d6ae80f1174ab7b96b64"
server_key_version: "encryptionkey@0"
I20260926 22:44:34.310318 16540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:34.318156 16540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:34.320151 16540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:34.320261 16540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:34.320724 16540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:34.320811 16540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.320875 16540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:34.320910 16540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:34.327893 16540 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:34683
I20260926 22:44:34.327948 16670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:34683 every 8 connection(s)
I20260926 22:44:34.328256 16540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:34.329012 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 16540
I20260926 22:44:34.329098 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:34.329267 15075 external_mini_cluster.cc:1428] Setting key 2d0a490038a1296da2b069bf4ddfff58
I20260926 22:44:34.331302 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.321084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:35755 (local address 127.14.184.254:40689)
0926 22:44:34.321129 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.321132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.321356 (+ 224us) server_negotiation.cc:407] Connection header received
0926 22:44:34.321833 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.321834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.321848 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.321910 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.322708 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.323569 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.325157 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.325510 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.325598 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.328345 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.328346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.328346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.328355 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.330065 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.330468 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.330469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.330469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.330525 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.330765 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.330766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.330766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.330909 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.330967 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.331174 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.331242 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:34.331460 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.321094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.14.184.195:35755)
0926 22:44:34.321236 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.321273 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.321726 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.321974 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.321978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.322277 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.322566 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.322571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.323679 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.323680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.325057 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.325060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.325619 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.325620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.325757 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.326462 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.326473 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.328241 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.330150 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.330156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.330158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.330387 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.330583 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.330583 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.330584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.330712 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.330960 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.330963 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.331065 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.331141 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.331252 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260926 22:44:34.331894 16671 heartbeater.cc:347] Connected to a master server at 127.14.184.254:40689
I20260926 22:44:34.331977 16671 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:34.332166 16671 heartbeater.cc:511] Master 127.14.184.254:40689 requested a full tablet report, sending...
I20260926 22:44:34.332536 16188 ts_manager.cc:194] Registered new tserver with Master: 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683)
I20260926 22:44:34.333060 16188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:35755
I20260926 22:44:34.338424 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.332939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:35797 (local address 127.14.184.254:40689)
0926 22:44:34.332974 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.332976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.333032 (+ 56us) server_negotiation.cc:407] Connection header received
0926 22:44:34.333094 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.333094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.333101 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.333162 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.333590 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.334054 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.334794 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.334996 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.335077 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.335960 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.335961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.335961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.335968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.337321 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.337661 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.337662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.337662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.337716 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.337928 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.337929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.337929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.338036 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.338110 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.338260 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.338334 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:44:34.338438 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.332846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.14.184.195:35797)
0926 22:44:34.332968 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.332979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.333039 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.333242 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.333243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.333449 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.333502 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.333504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.334148 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.334149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.334657 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.334659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.335079 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.335080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.335151 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.335580 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.335585 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.335854 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.337420 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.337421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.337421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.337589 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.337788 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.337789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.337789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.337846 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.338101 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.338102 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.338147 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.338235 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.338335 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260926 22:44:34.340931 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:34.362327 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.356322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.14.184.254:40689)
0926 22:44:34.356378 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.356380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.356387 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:34.356407 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.356407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.356414 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.356473 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.356932 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.357384 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.358055 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.358225 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.358272 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.360379 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.360380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.360380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.360386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.361524 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.361821 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.361822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.361822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.361867 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.362004 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.362005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.362005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.362081 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.362147 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.362221 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.362271 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:44:34.363686 16188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:34.365198 16188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:34.371244 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.367357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43037 (local address 127.0.0.1:53008)
0926 22:44:34.367660 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.367670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.367707 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.367975 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.367979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.367981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.368047 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.368049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.369096 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.369097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.370038 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.370040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.370941 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.370942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.371033 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.371047 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.371122 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.371170 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":59,"threads_started":1}
I20260926 22:44:34.371284 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.367598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.14.184.193:43037)
0926 22:44:34.367815 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.367818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.367826 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:34.367856 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.367858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.367870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.367966 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.368142 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.369020 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.370115 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.370861 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.370930 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.370984 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.371147 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.371187 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260926 22:44:34.371870 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.367356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38845 (local address 127.0.0.1:60184)
0926 22:44:34.367458 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.367496 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.367558 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.368160 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.368161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.368163 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.368213 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.368214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.369544 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.369545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.370722 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.370725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.371625 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.371626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.371689 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.371692 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.371750 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.371796 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"spinlock_wait_cycles":768,"thread_start_us":64,"threads_started":1}
I20260926 22:44:34.371891 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.367796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.14.184.194:38845)
0926 22:44:34.368014 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.368019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.368029 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:34.368064 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.368067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.368082 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.368163 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.368314 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.369452 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.370815 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.371524 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.371623 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.371673 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.371764 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.371799 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":117,"threads_started":1}
I20260926 22:44:34.372162 16325 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf9caa1935a433bb9362a6323c06281 (DEFAULT_TABLE table=test-table [id=9b82bfdfdbb84bd4b6fc658285362293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:34.372536 16325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf9caa1935a433bb9362a6323c06281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.372542 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.367578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.14.184.195:34683)
0926 22:44:34.367728 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.367732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.368320 (+ 588us) server_negotiation.cc:407] Connection header received
0926 22:44:34.368452 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.368455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.368470 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.368550 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.368651 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.369677 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.370650 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.371801 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.371870 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.372155 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.372397 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.372435 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260926 22:44:34.372555 16465 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf9caa1935a433bb9362a6323c06281 (DEFAULT_TABLE table=test-table [id=9b82bfdfdbb84bd4b6fc658285362293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:34.372721 16687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.367356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:34683 (local address 127.0.0.1:38948)
0926 22:44:34.368243 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.368253 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.368349 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.368558 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.368558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.368560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.368597 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.368598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.369762 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.369763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.370563 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.370565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.372193 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.372194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.372255 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.372259 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.372327 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.372655 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260926 22:44:34.372973 16465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf9caa1935a433bb9362a6323c06281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.375494 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf9caa1935a433bb9362a6323c06281 (DEFAULT_TABLE table=test-table [id=9b82bfdfdbb84bd4b6fc658285362293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:34.375643 16691 tablet_bootstrap.cc:492] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Bootstrap starting.
I20260926 22:44:34.375840 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf9caa1935a433bb9362a6323c06281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.376101 16692 tablet_bootstrap.cc:492] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Bootstrap starting.
I20260926 22:44:34.376430 16691 tablet_bootstrap.cc:654] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.376721 16692 tablet_bootstrap.cc:654] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.376776 16691 log.cc:824] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:34.377046 16692 log.cc:824] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:34.377596 16691 tablet_bootstrap.cc:492] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: No bootstrap required, opened a new log
I20260926 22:44:34.377672 16691 ts_tablet_manager.cc:1397] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:34.377812 16692 tablet_bootstrap.cc:492] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: No bootstrap required, opened a new log
I20260926 22:44:34.377889 16692 ts_tablet_manager.cc:1397] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:34.378930 16695 tablet_bootstrap.cc:492] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Bootstrap starting.
I20260926 22:44:34.379547 16691 raft_consensus.cc:348] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.379709 16691 raft_consensus.cc:374] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.379752 16691 raft_consensus.cc:729] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6378e119b24965b48f47d136285620, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.379899 16691 consensus_queue.cc:260] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.380007 16695 tablet_bootstrap.cc:654] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.379977 16692 raft_consensus.cc:348] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.380156 16692 raft_consensus.cc:374] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.380182 16691 ts_tablet_manager.cc:1428] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:44:34.380203 16692 raft_consensus.cc:729] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4352922687774180acaf347ec6631734, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.380323 16391 heartbeater.cc:503] Master 127.14.184.254:40689 was elected leader, sending a full tablet report...
I20260926 22:44:34.380316 16692 consensus_queue.cc:260] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.380358 16695 log.cc:824] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:34.380569 16692 ts_tablet_manager.cc:1428] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:34.380621 16531 heartbeater.cc:503] Master 127.14.184.254:40689 was elected leader, sending a full tablet report...
I20260926 22:44:34.381197 16695 tablet_bootstrap.cc:492] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: No bootstrap required, opened a new log
I20260926 22:44:34.381263 16695 ts_tablet_manager.cc:1397] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:34.382776 16695 raft_consensus.cc:348] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.382939 16695 raft_consensus.cc:374] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.382985 16695 raft_consensus.cc:729] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8e8388263c4d53a47a2576dbf62e6d, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.383095 16695 consensus_queue.cc:260] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.383361 16695 ts_tablet_manager.cc:1428] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:34.383440 16671 heartbeater.cc:503] Master 127.14.184.254:40689 was elected leader, sending a full tablet report...
W20260926 22:44:34.431895 16532 tablet.cc:2369] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:34.438452 16696 raft_consensus.cc:482] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:34.438570 16696 raft_consensus.cc:504] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.439019 16696 leader_election.cc:288] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4352922687774180acaf347ec6631734 (127.14.184.194:38845), 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683)
I20260926 22:44:34.442879 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.439314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:58241 (local address 127.14.184.195:34683)
0926 22:44:34.439363 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.439365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.439373 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:34.439403 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.439404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.439414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.439474 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.439671 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.440441 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.441903 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.442504 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.442561 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.442571 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.442770 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.442809 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 22:44:34.442879 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.439364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:45863 (local address 127.14.184.194:38845)
0926 22:44:34.439512 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.439514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.439521 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:34.439546 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.439547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.439558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.439618 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.439807 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.440497 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.441903 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.442491 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.442564 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.442571 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.442756 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.442808 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260926 22:44:34.442885 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.439204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:34683 (local address 127.14.184.193:58241)
0926 22:44:34.439264 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.439277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.439320 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.439499 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.439501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.439503 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.439582 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.439584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.440556 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.440557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.441813 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.441816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.442574 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.442575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.442699 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.442703 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.442756 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.442790 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":26}
I20260926 22:44:34.442905 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.439234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38845 (local address 127.14.184.193:45863)
0926 22:44:34.439417 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.439426 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.439462 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.439650 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.439651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.439653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.439724 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.439725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.440563 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.440564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.441813 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.441816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.442570 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.442571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.442655 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.442662 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.442734 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.442822 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":61,"thread_start_us":117,"threads_started":1}
I20260926 22:44:34.443276 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf9caa1935a433bb9362a6323c06281" candidate_uuid: "6e6378e119b24965b48f47d136285620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" is_pre_election: true
I20260926 22:44:34.443434 16625 raft_consensus.cc:2454] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e6378e119b24965b48f47d136285620 in term 0.
I20260926 22:44:34.443527 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf9caa1935a433bb9362a6323c06281" candidate_uuid: "6e6378e119b24965b48f47d136285620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4352922687774180acaf347ec6631734" is_pre_election: true
I20260926 22:44:34.443663 16485 raft_consensus.cc:2454] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e6378e119b24965b48f47d136285620 in term 0.
I20260926 22:44:34.443887 16276 leader_election.cc:302] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8e8388263c4d53a47a2576dbf62e6d, 6e6378e119b24965b48f47d136285620; no voters:
I20260926 22:44:34.444010 16696 raft_consensus.cc:2781] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:34.444061 16696 raft_consensus.cc:482] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:34.444113 16696 raft_consensus.cc:3037] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.445127 16696 raft_consensus.cc:504] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.445266 16696 leader_election.cc:288] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [CANDIDATE]: Term 1 election: Requested vote from peers 4352922687774180acaf347ec6631734 (127.14.184.194:38845), 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683)
I20260926 22:44:34.445408 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf9caa1935a433bb9362a6323c06281" candidate_uuid: "6e6378e119b24965b48f47d136285620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4352922687774180acaf347ec6631734"
I20260926 22:44:34.445492 16485 raft_consensus.cc:3037] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.445452 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf9caa1935a433bb9362a6323c06281" candidate_uuid: "6e6378e119b24965b48f47d136285620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8e8388263c4d53a47a2576dbf62e6d"
I20260926 22:44:34.445581 16625 raft_consensus.cc:3037] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.446257 16485 raft_consensus.cc:2454] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e6378e119b24965b48f47d136285620 in term 1.
I20260926 22:44:34.446305 16625 raft_consensus.cc:2454] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e6378e119b24965b48f47d136285620 in term 1.
I20260926 22:44:34.446465 16278 leader_election.cc:302] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4352922687774180acaf347ec6631734, 6e6378e119b24965b48f47d136285620; no voters:
I20260926 22:44:34.446563 16696 raft_consensus.cc:2781] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:34.446751 16696 raft_consensus.cc:686] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 1 LEADER]: Becoming Leader. State: Replica: 6e6378e119b24965b48f47d136285620, State: Running, Role: LEADER
I20260926 22:44:34.446853 16696 consensus_queue.cc:237] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.447664 16188 catalog_manager.cc:5495] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e6378e119b24965b48f47d136285620 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "6e6378e119b24965b48f47d136285620" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:34.465456 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.461993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.14.184.193:43037)
0926 22:44:34.462038 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.462040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.462216 (+ 176us) server_negotiation.cc:407] Connection header received
0926 22:44:34.462323 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.462324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.462343 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.462436 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:34.462625 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.463414 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.464282 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.464587 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.464653 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.464934 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.465060 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.465141 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.465328 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.465356 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:34.467530 16625 raft_consensus.cc:1261] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Refusing update from remote peer 6e6378e119b24965b48f47d136285620: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:34.467538 16485 raft_consensus.cc:1261] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Refusing update from remote peer 6e6378e119b24965b48f47d136285620: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:34.467870 16701 consensus_queue.cc:1035] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:34.468007 16696 consensus_queue.cc:1035] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:34.473091 16707 mvcc.cc:205] Tried to move back new op lower bound from 7333735114616242176 to 7333735114535022592. Current Snapshot: MvccSnapshot[applied={T|T < 7333735114616242176 or (T in {7333735114616242176})}]
I20260926 22:44:34.473202 16705 mvcc.cc:205] Tried to move back new op lower bound from 7333735114616242176 to 7333735114535022592. Current Snapshot: MvccSnapshot[applied={T|T < 7333735114616242176 or (T in {7333735114616242176})}]
I20260926 22:44:34.474920 16188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:34.475018 16188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:34.475744 16188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: Sending DeleteTablet for 3 replicas of tablet 5cf9caa1935a433bb9362a6323c06281
I20260926 22:44:34.476082 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf9caa1935a433bb9362a6323c06281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:34 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38948
I20260926 22:44:34.476109 16325 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf9caa1935a433bb9362a6323c06281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:34 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:53008
I20260926 22:44:34.476318 16465 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf9caa1935a433bb9362a6323c06281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:34 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:60184
I20260926 22:44:34.476356 16715 tablet_replica.cc:330] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: stopping tablet replica
I20260926 22:44:34.476527 16714 tablet_replica.cc:330] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: stopping tablet replica
I20260926 22:44:34.476534 16715 raft_consensus.cc:2229] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:34.476619 16714 raft_consensus.cc:2229] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:34.476570 16188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 22:44:34.476708 16714 raft_consensus.cc:2258] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 22:44:34.476712 16188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:34.477085 16716 tablet_replica.cc:330] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: stopping tablet replica
I20260926 22:44:34.477180 16716 raft_consensus.cc:2229] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:34.477304 16716 raft_consensus.cc:2258] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:34.477732 16716 ts_tablet_manager.cc:1905] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:34.477115 16715 raft_consensus.cc:2258] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:34.478276 16325 tablet_service.cc:1467] Processing CreateTablet for tablet 3e192fa208b7471ba8571dc3a6226cea (DEFAULT_TABLE table=test-table [id=586d1100ed9c4eeab25240d4ef961595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:34.478282 16715 ts_tablet_manager.cc:1905] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:34.478394 16325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e192fa208b7471ba8571dc3a6226cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.479434 16716 ts_tablet_manager.cc:1918] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:34.479496 16716 log.cc:1196] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/5cf9caa1935a433bb9362a6323c06281
I20260926 22:44:34.479743 16716 ts_tablet_manager.cc:1939] T 5cf9caa1935a433bb9362a6323c06281 P 4352922687774180acaf347ec6631734: Deleting consensus metadata
I20260926 22:44:34.479915 16465 tablet_service.cc:1467] Processing CreateTablet for tablet 3e192fa208b7471ba8571dc3a6226cea (DEFAULT_TABLE table=test-table [id=586d1100ed9c4eeab25240d4ef961595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:34.480018 16465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e192fa208b7471ba8571dc3a6226cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.480357 16174 catalog_manager.cc:4841] TS 4352922687774180acaf347ec6631734 (127.14.184.194:38845): tablet 5cf9caa1935a433bb9362a6323c06281 (table test-table [id=9b82bfdfdbb84bd4b6fc658285362293]) successfully deleted
I20260926 22:44:34.480682 16715 ts_tablet_manager.cc:1918] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:34.480744 16715 log.cc:1196] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/5cf9caa1935a433bb9362a6323c06281
I20260926 22:44:34.480980 16715 ts_tablet_manager.cc:1939] T 5cf9caa1935a433bb9362a6323c06281 P 6a8e8388263c4d53a47a2576dbf62e6d: Deleting consensus metadata
I20260926 22:44:34.481236 16692 tablet_bootstrap.cc:492] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Bootstrap starting.
I20260926 22:44:34.481346 16691 tablet_bootstrap.cc:492] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Bootstrap starting.
I20260926 22:44:34.481635 16692 tablet_bootstrap.cc:654] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.481650 16691 tablet_bootstrap.cc:654] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.482136 16172 catalog_manager.cc:4841] TS 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683): tablet 5cf9caa1935a433bb9362a6323c06281 (table test-table [id=9b82bfdfdbb84bd4b6fc658285362293]) successfully deleted
I20260926 22:44:34.482185 16692 tablet_bootstrap.cc:492] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: No bootstrap required, opened a new log
I20260926 22:44:34.482226 16692 ts_tablet_manager.cc:1397] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:34.482230 16691 tablet_bootstrap.cc:492] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: No bootstrap required, opened a new log
I20260926 22:44:34.482273 16691 ts_tablet_manager.cc:1397] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:34.482390 16692 raft_consensus.cc:348] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.482460 16692 raft_consensus.cc:374] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.482426 16691 raft_consensus.cc:348] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.482478 16692 raft_consensus.cc:729] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4352922687774180acaf347ec6631734, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.482488 16691 raft_consensus.cc:374] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.482509 16691 raft_consensus.cc:729] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6378e119b24965b48f47d136285620, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.482512 16692 consensus_queue.cc:260] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.482550 16691 consensus_queue.cc:260] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.482604 16692 ts_tablet_manager.cc:1428] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:34.482638 16691 ts_tablet_manager.cc:1428] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.482820 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 3e192fa208b7471ba8571dc3a6226cea (DEFAULT_TABLE table=test-table [id=586d1100ed9c4eeab25240d4ef961595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:34.482921 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e192fa208b7471ba8571dc3a6226cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.484004 16695 tablet_bootstrap.cc:492] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Bootstrap starting.
I20260926 22:44:34.484413 16695 tablet_bootstrap.cc:654] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.484824 16714 ts_tablet_manager.cc:1905] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:34.485038 16695 tablet_bootstrap.cc:492] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: No bootstrap required, opened a new log
I20260926 22:44:34.485091 16695 ts_tablet_manager.cc:1397] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:34.485220 16695 raft_consensus.cc:348] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.485299 16695 raft_consensus.cc:374] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.485320 16695 raft_consensus.cc:729] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8e8388263c4d53a47a2576dbf62e6d, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.485364 16695 consensus_queue.cc:260] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.485445 16695 ts_tablet_manager.cc:1428] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:34.486325 16714 ts_tablet_manager.cc:1918] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:34.486395 16714 log.cc:1196] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/5cf9caa1935a433bb9362a6323c06281
I20260926 22:44:34.486639 16714 ts_tablet_manager.cc:1939] T 5cf9caa1935a433bb9362a6323c06281 P 6e6378e119b24965b48f47d136285620: Deleting consensus metadata
I20260926 22:44:34.487061 16173 catalog_manager.cc:4841] TS 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037): tablet 5cf9caa1935a433bb9362a6323c06281 (table test-table [id=9b82bfdfdbb84bd4b6fc658285362293]) successfully deleted
W20260926 22:44:34.538827 16392 tablet.cc:2369] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:34.573154 16697 raft_consensus.cc:482] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:34.573269 16697 raft_consensus.cc:504] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.573639 16697 leader_election.cc:288] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037), 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683)
I20260926 22:44:34.577328 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.573899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:33627 (local address 127.14.184.193:43037)
0926 22:44:34.573944 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.573946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.573956 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:34.574050 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.574051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.574064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.574114 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.574331 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.575090 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.576581 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.576945 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.577034 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.577043 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.577207 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.577247 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:44:34.577328 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.573835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43037 (local address 127.14.184.194:33627)
0926 22:44:34.573894 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.573906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.573952 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.574162 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.574163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.574165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.574234 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.574236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.575171 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.575172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.576451 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.576455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.577034 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.577035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.577126 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.577129 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.577199 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.577247 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":47}
I20260926 22:44:34.577548 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.574053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:43263 (local address 127.14.184.195:34683)
0926 22:44:34.574099 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.574101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.574112 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:34.574167 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.574168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.574180 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.574212 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.574439 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.575015 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.576574 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.577125 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.577203 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.577211 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.577435 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.577478 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:34.577812 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.573856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:34683 (local address 127.14.184.194:43263)
0926 22:44:34.574049 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.574059 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.574102 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.574274 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.574274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.574276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.574363 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.574364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.575110 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.575112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.576452 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.576455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.577264 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.577266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.577354 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.577358 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.577688 (+ 330us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.577731 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":97,"thread_start_us":62,"threads_started":1}
I20260926 22:44:34.577759 16345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e192fa208b7471ba8571dc3a6226cea" candidate_uuid: "4352922687774180acaf347ec6631734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e6378e119b24965b48f47d136285620" is_pre_election: true
I20260926 22:44:34.577906 16345 raft_consensus.cc:2454] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4352922687774180acaf347ec6631734 in term 0.
I20260926 22:44:34.578006 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e192fa208b7471ba8571dc3a6226cea" candidate_uuid: "4352922687774180acaf347ec6631734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" is_pre_election: true
I20260926 22:44:34.578087 16625 raft_consensus.cc:2454] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4352922687774180acaf347ec6631734 in term 0.
I20260926 22:44:34.578121 16417 leader_election.cc:302] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4352922687774180acaf347ec6631734, 6e6378e119b24965b48f47d136285620; no voters:
I20260926 22:44:34.578255 16697 raft_consensus.cc:2781] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:34.578292 16697 raft_consensus.cc:482] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:34.578328 16697 raft_consensus.cc:3037] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Advancing to term 1
W20260926 22:44:34.578838 16672 tablet.cc:2369] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:34.579216 16697 raft_consensus.cc:504] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.579375 16697 leader_election.cc:288] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [CANDIDATE]: Term 1 election: Requested vote from peers 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037), 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683)
I20260926 22:44:34.579522 16345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e192fa208b7471ba8571dc3a6226cea" candidate_uuid: "4352922687774180acaf347ec6631734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e6378e119b24965b48f47d136285620"
I20260926 22:44:34.579628 16345 raft_consensus.cc:3037] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.579605 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e192fa208b7471ba8571dc3a6226cea" candidate_uuid: "4352922687774180acaf347ec6631734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8e8388263c4d53a47a2576dbf62e6d"
I20260926 22:44:34.579674 16625 raft_consensus.cc:3037] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.580281 16345 raft_consensus.cc:2454] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4352922687774180acaf347ec6631734 in term 1.
I20260926 22:44:34.580452 16417 leader_election.cc:302] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4352922687774180acaf347ec6631734, 6e6378e119b24965b48f47d136285620; no voters:
I20260926 22:44:34.580470 16625 raft_consensus.cc:2454] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4352922687774180acaf347ec6631734 in term 1.
I20260926 22:44:34.580586 16697 raft_consensus.cc:2781] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:34.580750 16697 raft_consensus.cc:686] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 1 LEADER]: Becoming Leader. State: Replica: 4352922687774180acaf347ec6631734, State: Running, Role: LEADER
I20260926 22:44:34.580817 16697 consensus_queue.cc:237] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.581542 16188 catalog_manager.cc:5495] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4352922687774180acaf347ec6631734 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "4352922687774180acaf347ec6631734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } health_report { overall_health: UNKNOWN } } }
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462673, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:34.622206 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.615366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40689 (local address 127.0.0.1:38032)
0926 22:44:34.615566 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.615576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.615641 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.615768 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.615769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.616069 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:34.616113 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.616115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.616893 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.616894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.617469 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.617471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.617817 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.617818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.617859 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.618489 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:34.618493 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:34.620016 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:34.621319 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.621325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.621327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.621511 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.621677 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:34.621678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:34.621678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:34.621731 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:34.621927 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:34.621933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:34.621972 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.622023 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.622106 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
I20260926 22:44:34.622253 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.615438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.14.184.254:40689)
0926 22:44:34.615519 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.615521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.615625 (+ 104us) server_negotiation.cc:407] Connection header received
0926 22:44:34.615683 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.615684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.615692 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.615741 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.616175 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.616792 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.617535 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.617752 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.617807 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.620087 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.620088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.620089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.620094 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.621244 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.621573 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.621574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.621574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.621621 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.621782 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.621783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.621783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.621875 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.621923 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.622053 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.622147 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":53}
I20260926 22:44:34.624033 16188 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38032:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 22:44:34.624294 16188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:34.625681 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 92c0f656fcca43b48893bd2bf90417c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5890bb625de54df786dac5b42c618b44]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:34.625716 16325 tablet_service.cc:1467] Processing CreateTablet for tablet 92c0f656fcca43b48893bd2bf90417c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5890bb625de54df786dac5b42c618b44]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:34.625893 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c0f656fcca43b48893bd2bf90417c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.625932 16325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c0f656fcca43b48893bd2bf90417c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.626988 16695 tablet_bootstrap.cc:492] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: Bootstrap starting.
I20260926 22:44:34.627169 16691 tablet_bootstrap.cc:492] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: Bootstrap starting.
I20260926 22:44:34.627440 16695 tablet_bootstrap.cc:654] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.627468 16691 tablet_bootstrap.cc:654] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.628026 16691 tablet_bootstrap.cc:492] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: No bootstrap required, opened a new log
I20260926 22:44:34.628103 16691 ts_tablet_manager.cc:1397] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:34.628212 16691 raft_consensus.cc:348] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.628278 16691 raft_consensus.cc:374] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.628293 16691 raft_consensus.cc:729] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6378e119b24965b48f47d136285620, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.628350 16691 consensus_queue.cc:260] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.628439 16691 ts_tablet_manager.cc:1428] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.625705 16465 tablet_service.cc:1467] Processing CreateTablet for tablet 92c0f656fcca43b48893bd2bf90417c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5890bb625de54df786dac5b42c618b44]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:34.628455 16701 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.628539 16701 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.628551 16465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c0f656fcca43b48893bd2bf90417c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:34.628750 16695 tablet_bootstrap.cc:492] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: No bootstrap required, opened a new log
I20260926 22:44:34.628790 16695 ts_tablet_manager.cc:1397] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:34.628953 16695 raft_consensus.cc:348] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.629019 16695 raft_consensus.cc:374] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.629033 16695 raft_consensus.cc:729] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8e8388263c4d53a47a2576dbf62e6d, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.629086 16695 consensus_queue.cc:260] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.629181 16695 ts_tablet_manager.cc:1428] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:34.629249 16699 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.629329 16699 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.630484 16692 tablet_bootstrap.cc:492] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: Bootstrap starting.
I20260926 22:44:34.630914 16692 tablet_bootstrap.cc:654] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:34.631495 16692 tablet_bootstrap.cc:492] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: No bootstrap required, opened a new log
I20260926 22:44:34.631536 16692 ts_tablet_manager.cc:1397] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:34.631688 16692 raft_consensus.cc:348] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.631742 16692 raft_consensus.cc:374] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:34.631755 16692 raft_consensus.cc:729] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4352922687774180acaf347ec6631734, State: Initialized, Role: FOLLOWER
I20260926 22:44:34.631819 16692 consensus_queue.cc:260] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.631915 16692 ts_tablet_manager.cc:1428] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:34.632053 16697 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.632126 16697 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.827812 16699 raft_consensus.cc:482] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:34.827907 16699 raft_consensus.cc:504] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.828200 16699 leader_election.cc:288] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037), 4352922687774180acaf347ec6631734 (127.14.184.194:38845)
I20260926 22:44:34.831077 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.828391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38845 (local address 127.14.184.195:58189)
0926 22:44:34.828461 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.828475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.828540 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.828697 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.828698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.828699 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.828777 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.828778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.829437 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.829438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.830219 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.830221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.830812 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.830813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.830896 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.830900 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.830959 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.831002 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 22:44:34.831113 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.828467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:58189 (local address 127.14.184.194:38845)
0926 22:44:34.828519 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.828521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.828532 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:34.828610 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.828611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.828622 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.828694 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.828853 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.829348 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.830318 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.830709 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.830812 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.830822 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.830978 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.831049 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 22:44:34.831357 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0f656fcca43b48893bd2bf90417c8" candidate_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4352922687774180acaf347ec6631734" is_pre_election: true
I20260926 22:44:34.831467 16485 raft_consensus.cc:2454] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8e8388263c4d53a47a2576dbf62e6d in term 0.
I20260926 22:44:34.831672 16558 leader_election.cc:302] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4352922687774180acaf347ec6631734, 6a8e8388263c4d53a47a2576dbf62e6d; no voters:
I20260926 22:44:34.831837 16699 raft_consensus.cc:2781] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:34.831851 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.828391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43037 (local address 127.14.184.195:42783)
0926 22:44:34.828529 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.828539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.828623 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.828783 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.828784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.828785 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:34.828876 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.828877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.829586 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.829588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.830859 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.830862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.831576 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.831578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.831676 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.831679 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.831724 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.831749 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":18,"thread_start_us":61,"threads_started":1}
I20260926 22:44:34.831878 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.828480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:42783 (local address 127.14.184.193:43037)
0926 22:44:34.828539 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.828542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.828607 (+ 65us) server_negotiation.cc:407] Connection header received
0926 22:44:34.828700 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.828701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.828710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.828780 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:34.828945 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.829506 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.830943 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.831448 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.831538 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.831546 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.831745 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.831800 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:44:34.831897 16699 raft_consensus.cc:482] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:34.831923 16699 raft_consensus.cc:3037] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.832140 16345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0f656fcca43b48893bd2bf90417c8" candidate_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e6378e119b24965b48f47d136285620" is_pre_election: true
I20260926 22:44:34.832229 16345 raft_consensus.cc:2454] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8e8388263c4d53a47a2576dbf62e6d in term 0.
I20260926 22:44:34.832780 16699 raft_consensus.cc:504] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.832914 16699 leader_election.cc:288] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [CANDIDATE]: Term 1 election: Requested vote from peers 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037), 4352922687774180acaf347ec6631734 (127.14.184.194:38845)
I20260926 22:44:34.833060 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0f656fcca43b48893bd2bf90417c8" candidate_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4352922687774180acaf347ec6631734"
I20260926 22:44:34.833149 16485 raft_consensus.cc:3037] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.833745 16485 raft_consensus.cc:2454] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8e8388263c4d53a47a2576dbf62e6d in term 1.
I20260926 22:44:34.833882 16345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0f656fcca43b48893bd2bf90417c8" candidate_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e6378e119b24965b48f47d136285620"
I20260926 22:44:34.833942 16558 leader_election.cc:302] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4352922687774180acaf347ec6631734, 6a8e8388263c4d53a47a2576dbf62e6d; no voters:
I20260926 22:44:34.833983 16345 raft_consensus.cc:3037] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:34.834036 16699 raft_consensus.cc:2781] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:34.834218 16699 raft_consensus.cc:686] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 LEADER]: Becoming Leader. State: Replica: 6a8e8388263c4d53a47a2576dbf62e6d, State: Running, Role: LEADER
I20260926 22:44:34.834295 16699 consensus_queue.cc:237] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } }
I20260926 22:44:34.834656 16733 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: TxnStatusTablet state changed. Reason: New leader 6a8e8388263c4d53a47a2576dbf62e6d. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.834781 16733 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:34.834791 16345 raft_consensus.cc:2454] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8e8388263c4d53a47a2576dbf62e6d in term 1.
I20260926 22:44:34.836915 16735 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 22:44:34.837042 16735 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 22:44:34.837046 16187 catalog_manager.cc:5495] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195). New cstate: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } health_report { overall_health: HEALTHY } } }
I20260926 22:44:34.856654 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.853952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.14.184.195:34683)
0926 22:44:34.854001 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.854003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.854012 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:34.854102 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.854103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.854122 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.854199 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:34.854357 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.854915 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.855817 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.856027 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.856101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.856292 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.856419 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.856487 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.856562 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.856580 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:44:34.856664 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.853884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:34683 (local address 127.0.0.1:38958)
0926 22:44:34.853957 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.853970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.854020 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.854203 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.854204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.854206 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:34.854255 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.854258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.855020 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.855021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.855726 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.855728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.856101 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.856102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.856169 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.856191 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.856476 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.856486 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.856529 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.856597 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 22:44:34.857882 16345 raft_consensus.cc:1261] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Refusing update from remote peer 6a8e8388263c4d53a47a2576dbf62e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:34.858107 16485 raft_consensus.cc:1261] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Refusing update from remote peer 6a8e8388263c4d53a47a2576dbf62e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:34.858382 16733 consensus_queue.cc:1035] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:34.858521 16699 consensus_queue.cc:1035] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:34.859124 16701 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: TxnStatusTablet state changed. Reason: New leader 6a8e8388263c4d53a47a2576dbf62e6d. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.859181 16697 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: TxnStatusTablet state changed. Reason: New leader 6a8e8388263c4d53a47a2576dbf62e6d. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.859244 16701 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.859268 16697 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.860014 16733 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.860149 16733 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:34.860275 16733 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.860401 16733 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6a8e8388263c4d53a47a2576dbf62e6d: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:34.860639 16704 mvcc.cc:205] Tried to move back new op lower bound from 7333735116215468032 to 7333735116121710592. Current Snapshot: MvccSnapshot[applied={T|T < 7333735116215468032}]
I20260926 22:44:34.860740 16701 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.860822 16701 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 6e6378e119b24965b48f47d136285620: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.861289 16697 tablet_replica.cc:439] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 } } peers { permanent_uuid: "4352922687774180acaf347ec6631734" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38845 } } peers { permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 } } }
I20260926 22:44:34.861368 16697 tablet_replica.cc:442] T 92c0f656fcca43b48893bd2bf90417c8 P 4352922687774180acaf347ec6631734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:34.864123 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.861801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.14.184.194:38845)
0926 22:44:34.861841 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.861843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.861865 (+ 22us) server_negotiation.cc:407] Connection header received
0926 22:44:34.861939 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.861940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.861962 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.862023 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:34.862147 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.862579 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.863369 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.863564 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.863639 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.863851 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.863927 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.863970 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.864038 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.864053 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:44:34.868849 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.866328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:54199 (local address 127.14.184.195:34683)
0926 22:44:34.866365 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.866367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.866553 (+ 186us) server_negotiation.cc:407] Connection header received
0926 22:44:34.866598 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.866599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.866605 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.866644 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:34.866777 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.867201 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.868243 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.868422 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.868468 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.868640 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.868664 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.868700 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.868791 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.868808 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:34.868918 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.866246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:34683 (local address 127.14.184.194:54199)
0926 22:44:34.866449 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.866459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.866548 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.866656 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.866656 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.866658 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:34.866729 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.866730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.867270 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.867271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.868165 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.868167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.868488 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.868489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.868552 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.868565 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.868718 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.868720 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.868795 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.868836 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260926 22:44:34.870824 16345 raft_consensus.cc:1261] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Refusing update from remote peer 4352922687774180acaf347ec6631734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:34.870832 16625 raft_consensus.cc:1261] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Refusing update from remote peer 4352922687774180acaf347ec6631734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:34.871047 16697 consensus_queue.cc:1035] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e6378e119b24965b48f47d136285620" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:34.871361 16721 consensus_queue.cc:1035] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8e8388263c4d53a47a2576dbf62e6d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 34683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:34.885640 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.876666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38845 (local address 127.14.184.195:44201)
0926 22:44:34.882800 (+ 6134us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:34.882812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:34.882886 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:34.883074 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:34.883076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:34.883077 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:34.883158 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.883160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.884085 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.884086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.884751 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:34.884753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.885218 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.885218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:34.885266 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.885279 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.885479 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.885481 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:44:34.885536 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:44:34.885571 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6094,"thread_start_us":70,"threads_started":1}
I20260926 22:44:34.885641 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.876704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:44201 (local address 127.14.184.194:38845)
0926 22:44:34.876742 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.876744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.882874 (+ 6130us) server_negotiation.cc:407] Connection header received
0926 22:44:34.882937 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.882938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.882949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.883003 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:34.883259 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.884007 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.884831 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.885138 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.885205 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.885363 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:34.885397 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:34.885469 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.885543 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.885564 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:44:34.897187 16188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:34.897302 16188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:34.897977 16188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca7f3520cf5746ec8b5491ad727d8340: Sending DeleteTablet for 3 replicas of tablet 3e192fa208b7471ba8571dc3a6226cea
I20260926 22:44:34.898264 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e192fa208b7471ba8571dc3a6226cea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:34 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38948
I20260926 22:44:34.898305 16325 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e192fa208b7471ba8571dc3a6226cea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:34 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:53008
I20260926 22:44:34.898356 16715 tablet_replica.cc:330] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: stopping tablet replica
I20260926 22:44:34.898391 16714 tablet_replica.cc:330] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: stopping tablet replica
I20260926 22:44:34.898401 16715 raft_consensus.cc:2229] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:34.898433 16715 raft_consensus.cc:2258] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:34.898437 16714 raft_consensus.cc:2229] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:34.898470 16714 raft_consensus.cc:2258] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:34.898912 16714 ts_tablet_manager.cc:1905] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:34.900491 16714 ts_tablet_manager.cc:1918] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:34.900557 16714 log.cc:1196] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/3e192fa208b7471ba8571dc3a6226cea
I20260926 22:44:34.900766 16714 ts_tablet_manager.cc:1939] T 3e192fa208b7471ba8571dc3a6226cea P 6e6378e119b24965b48f47d136285620: Deleting consensus metadata
I20260926 22:44:34.901094 16173 catalog_manager.cc:4841] TS 6e6378e119b24965b48f47d136285620 (127.14.184.193:43037): tablet 3e192fa208b7471ba8571dc3a6226cea (table test-table [id=586d1100ed9c4eeab25240d4ef961595]) successfully deleted
I20260926 22:44:34.901304 16715 ts_tablet_manager.cc:1905] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:34.902741 16465 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e192fa208b7471ba8571dc3a6226cea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:34 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:60184
I20260926 22:44:34.902853 16716 tablet_replica.cc:330] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: stopping tablet replica
I20260926 22:44:34.902913 16716 raft_consensus.cc:2229] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:34.903004 16716 raft_consensus.cc:2258] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:34.903810 16716 ts_tablet_manager.cc:1905] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:34.905299 16716 ts_tablet_manager.cc:1918] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:34.905359 16716 log.cc:1196] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/3e192fa208b7471ba8571dc3a6226cea
I20260926 22:44:34.905555 16716 ts_tablet_manager.cc:1939] T 3e192fa208b7471ba8571dc3a6226cea P 4352922687774180acaf347ec6631734: Deleting consensus metadata
I20260926 22:44:34.905913 16174 catalog_manager.cc:4841] TS 4352922687774180acaf347ec6631734 (127.14.184.194:38845): tablet 3e192fa208b7471ba8571dc3a6226cea (table test-table [id=586d1100ed9c4eeab25240d4ef961595]) successfully deleted
I20260926 22:44:34.906059 16715 ts_tablet_manager.cc:1918] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:34.906114 16715 log.cc:1196] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/3e192fa208b7471ba8571dc3a6226cea
I20260926 22:44:34.906324 16715 ts_tablet_manager.cc:1939] T 3e192fa208b7471ba8571dc3a6226cea P 6a8e8388263c4d53a47a2576dbf62e6d: Deleting consensus metadata
I20260926 22:44:34.906634 16172 catalog_manager.cc:4841] TS 6a8e8388263c4d53a47a2576dbf62e6d (127.14.184.195:34683): tablet 3e192fa208b7471ba8571dc3a6226cea (table test-table [id=586d1100ed9c4eeab25240d4ef961595]) successfully deleted
Sep 26 22:44:34 dist-test-slave-3986 krb5kdc[16143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462674, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:34.916826 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.903858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.14.184.193:43037)
0926 22:44:34.905809 (+ 1951us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.905812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.906655 (+ 843us) server_negotiation.cc:407] Connection header received
0926 22:44:34.906779 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.906780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.906801 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.906831 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.907560 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.908335 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.909367 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.909639 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.909679 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.913251 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.913262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.913262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.913270 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.915344 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.915848 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.915851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.915852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.915929 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.916136 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.916137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.916137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.916293 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.916403 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.916629 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.916717 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1915}
W20260926 22:44:34.917593 16385 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53026
I20260926 22:44:34.926060 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:34.918909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.14.184.254:40689)
0926 22:44:34.918952 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:34.918954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:34.919082 (+ 128us) server_negotiation.cc:407] Connection header received
0926 22:44:34.919175 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:34.919176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:34.919187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:34.919215 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:34.919900 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.920603 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.921552 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:34.921843 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:34.921887 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:34.923245 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:34.923246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:34.923247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:34.923253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:34.924827 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.925210 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.925211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.925211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.925267 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:34.925477 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:34.925478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:34.925478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:34.925600 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:34.925672 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:34.925896 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 22:44:34.925970 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 22:44:34.926559 16188 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38048
I20260926 22:44:34.927320 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 16260
I20260926 22:44:34.935914 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 16400
I20260926 22:44:34.941380 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 16540
I20260926 22:44:34.946861 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 16156
2026-09-26T22:44:34Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2378 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:35 dist-test-slave-3986 krb5kdc[16756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:35 dist-test-slave-3986 krb5kdc[16756](info): set up 2 sockets
Sep 26 22:44:35 dist-test-slave-3986 krb5kdc[16756](info): commencing operation
krb5kdc: starting...
W20260926 22:44:35.970536 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:36.081615 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38393
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:38393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:36.161398 16769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:36.161639 16769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:36.161675 16769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:36.162827 16769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:36.162892 16769 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 22:44:36.162911 16769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:36.162930 16769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:36.162976 16769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:36.162997 16769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:36.164427 16769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:38393
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:36.164705 16769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:36.164932 16769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:36.169696 16769 server_base.cc:1034] running on GCE node
W20260926 22:44:36.169680 16777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.169687 16774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.169670 16775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:36.170136 16769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:36.170425 16769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:36.171586 16769 hybrid_clock.cc:648] HybridClock initialized: now 1790462676171574 us; error 39 us; skew 500 ppm
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:36.173960 16769 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:36.174415 16769 webserver.cc:460] Webserver started at http://127.14.184.254:43439/ using document root <none> and password file <none>
I20260926 22:44:36.174652 16769 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:36.174785 16769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:36.174950 16769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:36.176034 16769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "4b95f529dd034b6b9e265a89efab2ac4"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "1d9b1a2a9ab63411c18df0d50b5ccbeb"
server_key_iv: "7ef35b5b3ffb41873a2e3c5531638aaa"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.176390 16769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "4b95f529dd034b6b9e265a89efab2ac4"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "1d9b1a2a9ab63411c18df0d50b5ccbeb"
server_key_iv: "7ef35b5b3ffb41873a2e3c5531638aaa"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.177693 16769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:36.178401 16784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.178563 16769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:36.178659 16769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "4b95f529dd034b6b9e265a89efab2ac4"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "1d9b1a2a9ab63411c18df0d50b5ccbeb"
server_key_iv: "7ef35b5b3ffb41873a2e3c5531638aaa"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.178768 16769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:36.190433 16769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:36.192178 16769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:36.192332 16769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:36.196991 16769 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:38393
I20260926 22:44:36.197048 16846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:38393 every 8 connection(s)
I20260926 22:44:36.197345 16769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:36.197901 16847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.200150 16847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Bootstrap starting.
I20260926 22:44:36.200714 16847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.200996 16847 log.cc:824] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:36.201632 16847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: No bootstrap required, opened a new log
I20260926 22:44:36.203827 16847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b95f529dd034b6b9e265a89efab2ac4" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38393 } }
I20260926 22:44:36.203958 16847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.203980 16847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b95f529dd034b6b9e265a89efab2ac4, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.204146 16847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b95f529dd034b6b9e265a89efab2ac4" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38393 } }
I20260926 22:44:36.204205 16847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:36.204245 16847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:36.204293 16847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.204782 16847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b95f529dd034b6b9e265a89efab2ac4" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38393 } }
I20260926 22:44:36.204897 16847 leader_election.cc:302] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b95f529dd034b6b9e265a89efab2ac4; no voters:
I20260926 22:44:36.205070 16847 leader_election.cc:288] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:36.205106 16850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:36.205263 16850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 4b95f529dd034b6b9e265a89efab2ac4, State: Running, Role: LEADER
I20260926 22:44:36.205333 16847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:36.205358 16850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b95f529dd034b6b9e265a89efab2ac4" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38393 } }
I20260926 22:44:36.205775 16852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b95f529dd034b6b9e265a89efab2ac4. Latest consensus state: current_term: 1 leader_uuid: "4b95f529dd034b6b9e265a89efab2ac4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b95f529dd034b6b9e265a89efab2ac4" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38393 } } }
I20260926 22:44:36.205876 16852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:36.205890 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 16769
I20260926 22:44:36.205966 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:36.206028 16851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b95f529dd034b6b9e265a89efab2ac4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b95f529dd034b6b9e265a89efab2ac4" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38393 } } }
I20260926 22:44:36.206103 16851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:36.206156 15075 external_mini_cluster.cc:1428] Setting key 37b13000b09c1e3beba7daff2176e1c1
I20260926 22:44:36.206624 16858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:36.207271 16858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:36.208879 16858 catalog_manager.cc:1347] Generated new cluster ID: fa14e71c5fbc43f283f7681eacaa166c
I20260926 22:44:36.208928 16858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:36.217592 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.206573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.14.184.254:38393)
0926 22:44:36.206808 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.206817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.206851 (+ 34us) server_negotiation.cc:407] Connection header received
0926 22:44:36.207131 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.207138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.207196 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.208110 (+ 914us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.208178 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.209526 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.210578 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.211189 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.211538 (+ 349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.213858 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.213862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.213862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.213873 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.215806 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.216195 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.216197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.216198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.216287 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.216483 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.216483 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.216484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.216753 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.217196 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.217257 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.217376 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":108,"threads_started":1}
I20260926 22:44:36.220726 16858 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:36.221184 16858 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:36.230240 16858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Generated new TSK 0
I20260926 22:44:36.230445 16858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:36.259094 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38393
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:36.338605 16873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:36.338866 16873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:36.338891 16873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:36.340016 16873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:36.340056 16873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:36.340101 16873 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 22:44:36.340127 16873 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:36.340155 16873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:36.341575 16873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:38393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:36.341812 16873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:36.342041 16873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:36.346897 16881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.346971 16879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.346885 16878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:36.347038 16873 server_base.cc:1034] running on GCE node
I20260926 22:44:36.347211 16873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:36.347453 16873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:36.348589 16873 hybrid_clock.cc:648] HybridClock initialized: now 1790462676348574 us; error 29 us; skew 500 ppm
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:36.350940 16873 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:36.351302 16873 webserver.cc:460] Webserver started at http://127.14.184.193:35589/ using document root <none> and password file <none>
I20260926 22:44:36.351472 16873 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:36.351511 16873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:36.351596 16873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:36.352519 16873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "f8727cc594e844889b22d525801ae1c8"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "7d606b36051510959eccaebd3671b19c"
server_key_iv: "b175cedd97129af57e4ccaee73ea9ed1"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.352840 16873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "f8727cc594e844889b22d525801ae1c8"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "7d606b36051510959eccaebd3671b19c"
server_key_iv: "b175cedd97129af57e4ccaee73ea9ed1"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.354056 16873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:44:36.354774 16888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.354938 16873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:36.355020 16873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "f8727cc594e844889b22d525801ae1c8"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "7d606b36051510959eccaebd3671b19c"
server_key_iv: "b175cedd97129af57e4ccaee73ea9ed1"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.355072 16873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:36.367197 16873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:36.369375 16873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:36.369479 16873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:36.369902 16873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:36.369941 16873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.369966 16873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:36.370010 16873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.376510 16873 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:36373
I20260926 22:44:36.376583 17003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:36373 every 8 connection(s)
I20260926 22:44:36.376874 16873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:36.379720 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.370404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:43611 (local address 127.14.184.254:38393)
0926 22:44:36.370479 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.370481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.370630 (+ 149us) server_negotiation.cc:407] Connection header received
0926 22:44:36.371183 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.371184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.371200 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.371281 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.372101 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.372942 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.374534 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.374933 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.375020 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.377433 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.377434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.377435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.377443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.378574 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.378989 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.378990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.378990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.379054 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.379252 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.379252 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.379252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.379348 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.379397 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.379593 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.379658 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 22:44:36.379931 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.370359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.14.184.193:43611)
0926 22:44:36.370506 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.370546 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.371087 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.371387 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.371391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.371720 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.371994 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.371999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.373087 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.373088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.374432 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.374435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.375024 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.375025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.375124 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.375604 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.375614 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.377336 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.378643 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.378648 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.378650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.378924 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.379112 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.379113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.379113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.379201 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.379400 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.379404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.379505 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.379581 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.379689 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260926 22:44:36.380390 17004 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38393
I20260926 22:44:36.380463 17004 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:36.380637 17004 heartbeater.cc:511] Master 127.14.184.254:38393 requested a full tablet report, sending...
I20260926 22:44:36.381175 16801 ts_manager.cc:194] Registered new tserver with Master: f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373)
I20260926 22:44:36.382120 16801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:43611
I20260926 22:44:36.383335 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 16873
I20260926 22:44:36.383445 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:36.383634 15075 external_mini_cluster.cc:1428] Setting key 574a411c2f3f3abfb4e684971c5b9bb6
I20260926 22:44:36.387354 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.381443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40103 (local address 127.14.184.254:38393)
0926 22:44:36.381479 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.381481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.381616 (+ 135us) server_negotiation.cc:407] Connection header received
0926 22:44:36.381667 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.381668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.381676 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.381719 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.382059 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.382793 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.383550 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.383823 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.383921 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.384762 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.384763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.384764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.384771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.386237 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.386637 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.386637 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.386638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.386725 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.386919 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.386919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.386919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.387014 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.387065 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.387239 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.387299 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 22:44:36.387495 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.381391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.14.184.193:40103)
0926 22:44:36.381549 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.381559 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.381615 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.381730 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.381730 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.381909 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.381969 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.381971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.382890 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.382891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.383475 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.383477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.383929 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.383930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.383987 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.384379 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.384383 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.384675 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.386322 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.386323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.386324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.386565 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.386787 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.386787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.386788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.386858 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.387077 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.387078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.387142 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.387215 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.387341 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:36.407413 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38393
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:36.489959 17013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:36.490137 17013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:36.490161 17013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:36.491293 17013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:36.491331 17013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:36.491351 17013 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 22:44:36.491362 17013 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:36.491379 17013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:36.492676 17013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:38393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:36.492847 17013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:36.493011 17013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:36.497756 17019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.497762 17021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:36.497780 17013 server_base.cc:1034] running on GCE node
W20260926 22:44:36.497762 17018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:36.498180 17013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:36.498430 17013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:36.499593 17013 hybrid_clock.cc:648] HybridClock initialized: now 1790462676499588 us; error 37 us; skew 500 ppm
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:36.501878 17013 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:36.502354 17013 webserver.cc:460] Webserver started at http://127.14.184.194:44711/ using document root <none> and password file <none>
I20260926 22:44:36.502599 17013 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:36.502664 17013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:36.502821 17013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:36.503803 17013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "e04715c541de4699a33454169cb60c4a"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "0e4e20447d85321e328073683e28194a"
server_key_iv: "91dc1493602bee83c4b48b510b2241ae"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.504156 17013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "e04715c541de4699a33454169cb60c4a"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "0e4e20447d85321e328073683e28194a"
server_key_iv: "91dc1493602bee83c4b48b510b2241ae"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.505478 17013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:36.506175 17028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.506354 17013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:36.506486 17013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "e04715c541de4699a33454169cb60c4a"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "0e4e20447d85321e328073683e28194a"
server_key_iv: "91dc1493602bee83c4b48b510b2241ae"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.506579 17013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:36.514971 17013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:36.516637 17013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:36.516794 17013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:36.517329 17013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:36.517386 17013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.517426 17013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:36.517450 17013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:36.524809 17013 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:38953
I20260926 22:44:36.525130 17013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:36.525126 17143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:38953 every 8 connection(s)
I20260926 22:44:36.527408 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.517655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:46447 (local address 127.14.184.254:38393)
0926 22:44:36.517724 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.517726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.517912 (+ 186us) server_negotiation.cc:407] Connection header received
0926 22:44:36.518341 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.518343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.518355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.518437 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.519040 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.519887 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.520969 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.521207 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.521308 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.524524 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.524526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.524527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.524534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.526170 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.526533 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.526533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.526534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.526582 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.526839 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.526840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.526840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.526954 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.527005 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.527255 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.527333 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 22:44:36.527530 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.517645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.14.184.194:46447)
0926 22:44:36.517782 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.517819 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.518248 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.518487 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.518490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.518766 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.518941 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.518945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.520004 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.520005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.520875 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.520877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.521320 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.521321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.521441 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.521921 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.521932 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.524427 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.526239 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.526243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.526245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.526455 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.526621 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.526621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.526621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.526773 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.527009 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.527014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.527130 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.527214 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.527363 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":83,"threads_started":1}
I20260926 22:44:36.528026 17144 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38393
I20260926 22:44:36.528105 17144 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:36.528270 17144 heartbeater.cc:511] Master 127.14.184.254:38393 requested a full tablet report, sending...
I20260926 22:44:36.528602 16801 ts_manager.cc:194] Registered new tserver with Master: e04715c541de4699a33454169cb60c4a (127.14.184.194:38953)
I20260926 22:44:36.529105 16801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:46447
I20260926 22:44:36.531543 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17013
I20260926 22:44:36.531642 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:36.531821 15075 external_mini_cluster.cc:1428] Setting key 24640a6e57af183418aa594214023360
I20260926 22:44:36.534271 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.528950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.14.184.194:59913)
0926 22:44:36.529072 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.529082 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.529144 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.529270 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.529270 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.529434 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.529485 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.529486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.530107 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.530108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.530636 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.530637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.531035 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.531036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.531110 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.531452 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.531456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.531757 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.533244 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.533245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.533245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.533478 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.533648 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.533649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.533649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.533710 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.534056 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.534056 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.534089 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.534141 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.534209 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260926 22:44:36.534343 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.529029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:59913 (local address 127.14.184.254:38393)
0926 22:44:36.529069 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.529070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.529134 (+ 64us) server_negotiation.cc:407] Connection header received
0926 22:44:36.529196 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.529197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.529207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.529265 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.529574 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.530015 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.530755 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.530953 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.531032 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.531839 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.531840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.531840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.531847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.533168 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.533549 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.533549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.533550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.533595 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.533776 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.533777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.533778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.533986 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.534059 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.534166 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.534241 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:36.555382 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38393
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:36.634483 17153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:36.634725 17153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:36.634752 17153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:36.635890 17153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:36.635929 17153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:36.635950 17153 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 22:44:36.635987 17153 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:36.636023 17153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:36.637436 17153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:38393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:36.637668 17153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:36.637895 17153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:36.642660 17158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.642727 17159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:36.642761 17161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:36.642911 17153 server_base.cc:1034] running on GCE node
I20260926 22:44:36.643106 17153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:36.643325 17153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:36.644488 17153 hybrid_clock.cc:648] HybridClock initialized: now 1790462676644476 us; error 24 us; skew 500 ppm
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:36.646857 17153 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:36.647249 17153 webserver.cc:460] Webserver started at http://127.14.184.195:34895/ using document root <none> and password file <none>
I20260926 22:44:36.647495 17153 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:36.647555 17153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:36.647691 17153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:36.648582 17153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "3cf210d17b5e4ba880615c98f4da3272"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "336f910ed59a198abb50a7f0e461d8e3"
server_key_iv: "a649e7b0bb0569b42509f6fbd573fb8d"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.649003 17153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "3cf210d17b5e4ba880615c98f4da3272"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "336f910ed59a198abb50a7f0e461d8e3"
server_key_iv: "a649e7b0bb0569b42509f6fbd573fb8d"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.650581 17153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:36.651526 17168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.651980 17153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:36.652114 17153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "3cf210d17b5e4ba880615c98f4da3272"
format_stamp: "Formatted at 2026-09-26 22:44:36 on dist-test-slave-3986"
server_key: "336f910ed59a198abb50a7f0e461d8e3"
server_key_iv: "a649e7b0bb0569b42509f6fbd573fb8d"
server_key_version: "encryptionkey@0"
I20260926 22:44:36.652184 17153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:36.663419 17153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:36.665068 17153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:36.665220 17153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:36.665684 17153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:36.665723 17153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.665745 17153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:36.665789 17153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:36.672627 17153 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:46419
I20260926 22:44:36.672706 17283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:46419 every 8 connection(s)
I20260926 22:44:36.673024 17153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:36.675128 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.666186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:53383 (local address 127.14.184.254:38393)
0926 22:44:36.666260 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.666263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.666405 (+ 142us) server_negotiation.cc:407] Connection header received
0926 22:44:36.666915 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.666916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.666931 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.667020 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.667510 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.668311 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.669311 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.669623 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.669690 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.672453 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.672455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.672456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.672464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.673996 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.674343 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.674344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.674344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.674391 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.674539 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.674540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.674540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.674647 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.674765 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.675001 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.675065 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 22:44:36.675282 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.666118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.14.184.195:53383)
0926 22:44:36.666277 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.666323 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.666817 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.667078 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.667080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.667276 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.667440 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.667444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.668410 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.668411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.669209 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.669211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.669703 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.669704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.669851 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.670374 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.670385 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.672353 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.674063 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.674067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.674069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.674269 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.674431 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.674431 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.674431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.674507 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.674759 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.674764 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.674886 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.674969 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.675080 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260926 22:44:36.675726 17284 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38393
I20260926 22:44:36.675806 17284 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:36.675999 17284 heartbeater.cc:511] Master 127.14.184.254:38393 requested a full tablet report, sending...
I20260926 22:44:36.676388 16800 ts_manager.cc:194] Registered new tserver with Master: 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:36.676910 16800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:53383
I20260926 22:44:36.679908 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17153
I20260926 22:44:36.680027 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:36.680224 15075 external_mini_cluster.cc:1428] Setting key 1945bb24ffb033a0917a8ddace4bf2c9
I20260926 22:44:36.681182 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:44:36.682554 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.676713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.14.184.195:56345)
0926 22:44:36.676855 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.676866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.676926 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.677070 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.677071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.677223 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.677272 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.677274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.678045 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.678046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.678530 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.678531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.678916 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.678917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.678973 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.679401 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.679405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.679659 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.681468 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.681468 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.681469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.681690 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.681923 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.681924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.681924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.682006 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.682275 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.682276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.682336 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.682411 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.682484 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260926 22:44:36.682658 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.676806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:56345 (local address 127.14.184.254:38393)
0926 22:44:36.676846 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.676848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.676914 (+ 66us) server_negotiation.cc:407] Connection header received
0926 22:44:36.676998 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.676999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.677009 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.677065 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.677334 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.677959 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.678593 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.678853 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.678906 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.679772 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.679773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.679773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.679779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.681396 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.681788 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.681789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.681789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.681854 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.682071 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.682072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.682073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.682211 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.682280 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.682447 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.682525 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:36.702729 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.696329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.14.184.254:38393)
0926 22:44:36.696390 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.696392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.696400 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:36.696423 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.696424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.696437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.696485 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.696986 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.697429 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.698097 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.698266 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.698316 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.700651 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.700652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.700653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.700658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.701848 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.702135 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.702136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.702136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.702180 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.702321 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.702321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.702321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.702402 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.702487 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.702565 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.702616 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:44:36.704159 16800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:36.705734 16800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:36.712042 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.708057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.14.184.195:46419)
0926 22:44:36.708442 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.708445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.708453 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:36.708490 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.708494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.708506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.708581 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.708736 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.709767 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711057 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.711704 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711780 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.711830 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.711919 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.711954 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":71,"threads_started":1}
I20260926 22:44:36.712038 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.707896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:36373 (local address 127.0.0.1:39466)
0926 22:44:36.708117 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.708128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.708178 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.708576 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.708577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.708580 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.708671 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.708673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.710002 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.710003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.710922 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.710924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.711784 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.711862 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.711866 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.711895 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.711937 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":37,"thread_start_us":55,"threads_started":1}
I20260926 22:44:36.712039 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.707896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:46419 (local address 127.0.0.1:52130)
0926 22:44:36.708046 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.708100 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.708164 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.708579 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.708580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.708582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.708662 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.708664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.709848 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.709849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.710984 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.710987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.711779 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.711854 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.711857 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.711908 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.711942 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":28,"thread_start_us":53,"threads_started":1}
I20260926 22:44:36.711966 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.707910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.0.0.1:39838)
0926 22:44:36.708255 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.708263 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.708301 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.708502 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.708505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.708507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.708568 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.708570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.709674 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.709676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.711002 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.711003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.711657 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.711734 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.711749 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.711823 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.711880 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":39,"thread_start_us":48,"threads_started":1}
I20260926 22:44:36.712427 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.708135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.14.184.194:38953)
0926 22:44:36.708327 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.708331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.708341 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:36.708378 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.708380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.708393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.708480 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.708650 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.709599 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711087 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.711561 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.711648 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.711704 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.712004 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.712046 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
I20260926 22:44:36.712610 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.708228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.14.184.193:36373)
0926 22:44:36.708397 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.708402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.708412 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:36.708452 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.708454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.708470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.708825 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.708881 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.709921 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.710982 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.711678 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.712391 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.712456 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.712481 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.712521 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260926 22:44:36.712890 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 23a2fed3e3d1465793a46cb04dcb853d (DEFAULT_TABLE table=test-table [id=d4400126d3da4a21a47807affeaa30b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:36.713048 17078 tablet_service.cc:1467] Processing CreateTablet for tablet 23a2fed3e3d1465793a46cb04dcb853d (DEFAULT_TABLE table=test-table [id=d4400126d3da4a21a47807affeaa30b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:36.713276 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a2fed3e3d1465793a46cb04dcb853d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.713389 17078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a2fed3e3d1465793a46cb04dcb853d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.715706 17304 tablet_bootstrap.cc:492] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Bootstrap starting.
I20260926 22:44:36.716346 17305 tablet_bootstrap.cc:492] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Bootstrap starting.
I20260926 22:44:36.716324 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 23a2fed3e3d1465793a46cb04dcb853d (DEFAULT_TABLE table=test-table [id=d4400126d3da4a21a47807affeaa30b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:36.716573 17304 tablet_bootstrap.cc:654] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.716681 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a2fed3e3d1465793a46cb04dcb853d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.716928 17304 log.cc:824] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:36.716954 17305 tablet_bootstrap.cc:654] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.717279 17305 log.cc:824] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:36.718123 17305 tablet_bootstrap.cc:492] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: No bootstrap required, opened a new log
I20260926 22:44:36.718199 17305 ts_tablet_manager.cc:1397] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:36.718981 17304 tablet_bootstrap.cc:492] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: No bootstrap required, opened a new log
I20260926 22:44:36.719061 17304 ts_tablet_manager.cc:1397] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 22:44:36.720497 17308 tablet_bootstrap.cc:492] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Bootstrap starting.
I20260926 22:44:36.720554 17305 raft_consensus.cc:348] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.720732 17305 raft_consensus.cc:374] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.720772 17305 raft_consensus.cc:729] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04715c541de4699a33454169cb60c4a, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.720901 17305 consensus_queue.cc:260] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.720999 17304 raft_consensus.cc:348] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.721150 17305 ts_tablet_manager.cc:1428] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:36.721163 17304 raft_consensus.cc:374] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.721159 17144 heartbeater.cc:503] Master 127.14.184.254:38393 was elected leader, sending a full tablet report...
I20260926 22:44:36.721210 17304 raft_consensus.cc:729] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf210d17b5e4ba880615c98f4da3272, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.721330 17304 consensus_queue.cc:260] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.721484 17308 tablet_bootstrap.cc:654] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.721591 17304 ts_tablet_manager.cc:1428] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:44:36.721657 17284 heartbeater.cc:503] Master 127.14.184.254:38393 was elected leader, sending a full tablet report...
I20260926 22:44:36.721786 17308 log.cc:824] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:36.722488 17308 tablet_bootstrap.cc:492] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: No bootstrap required, opened a new log
I20260926 22:44:36.722558 17308 ts_tablet_manager.cc:1397] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:36.724092 17308 raft_consensus.cc:348] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.724234 17308 raft_consensus.cc:374] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.724279 17308 raft_consensus.cc:729] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.724388 17308 consensus_queue.cc:260] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.724640 17308 ts_tablet_manager.cc:1428] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:36.724707 17004 heartbeater.cc:503] Master 127.14.184.254:38393 was elected leader, sending a full tablet report...
W20260926 22:44:36.775777 17145 tablet.cc:2369] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:36.787894 17312 raft_consensus.cc:482] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:36.788048 17312 raft_consensus.cc:504] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.788405 17312 leader_election.cc:288] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419), e04715c541de4699a33454169cb60c4a (127.14.184.194:38953)
I20260926 22:44:36.791360 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.788729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:55703 (local address 127.14.184.194:38953)
0926 22:44:36.788777 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.788779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.788791 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:44:36.788833 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.788834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.788844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.788919 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.789145 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.789637 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.790572 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.791006 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.791063 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.791070 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.791246 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.791290 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:44:36.791394 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.788709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:60199 (local address 127.14.184.195:46419)
0926 22:44:36.788790 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.788792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.788943 (+ 151us) server_negotiation.cc:407] Connection header received
0926 22:44:36.788981 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.788982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.788991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.789052 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.789229 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.789666 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.790593 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.791024 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.791113 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.791122 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.791290 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.791333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 22:44:36.791383 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.788627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.14.184.193:55703)
0926 22:44:36.788701 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.788713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.788765 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.788950 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.788952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.788954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.789056 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.789058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.789721 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.789722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.790473 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.790475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.791086 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.791087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.791176 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.791180 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.791222 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.791269 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 22:44:36.791539 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.788629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:46419 (local address 127.14.184.193:60199)
0926 22:44:36.788865 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.788875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.788934 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.789060 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.789061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.789062 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.789160 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.789161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.789767 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.789768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.790483 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.790485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.791131 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.791132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.791195 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.791198 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.791274 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.791316 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":50,"thread_start_us":126,"threads_started":1}
I20260926 22:44:36.791745 17086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2fed3e3d1465793a46cb04dcb853d" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04715c541de4699a33454169cb60c4a" is_pre_election: true
I20260926 22:44:36.791908 17086 raft_consensus.cc:2454] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 0.
I20260926 22:44:36.791939 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2fed3e3d1465793a46cb04dcb853d" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272" is_pre_election: true
I20260926 22:44:36.792109 17238 raft_consensus.cc:2454] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 0.
I20260926 22:44:36.792146 16892 leader_election.cc:302] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e04715c541de4699a33454169cb60c4a, f8727cc594e844889b22d525801ae1c8; no voters:
I20260926 22:44:36.792346 17312 raft_consensus.cc:2781] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:36.792387 17312 raft_consensus.cc:482] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:36.792410 17312 raft_consensus.cc:3037] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.793045 17312 raft_consensus.cc:504] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.793182 17312 leader_election.cc:288] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 election: Requested vote from peers 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419), e04715c541de4699a33454169cb60c4a (127.14.184.194:38953)
I20260926 22:44:36.793332 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2fed3e3d1465793a46cb04dcb853d" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272"
I20260926 22:44:36.793346 17086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2fed3e3d1465793a46cb04dcb853d" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04715c541de4699a33454169cb60c4a"
I20260926 22:44:36.793413 17086 raft_consensus.cc:3037] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.793414 17238 raft_consensus.cc:3037] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.794262 17086 raft_consensus.cc:2454] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 1.
I20260926 22:44:36.794262 17238 raft_consensus.cc:2454] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 1.
I20260926 22:44:36.794425 16892 leader_election.cc:302] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e04715c541de4699a33454169cb60c4a, f8727cc594e844889b22d525801ae1c8; no voters:
I20260926 22:44:36.794520 17312 raft_consensus.cc:2781] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:36.794655 17312 raft_consensus.cc:686] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 1 LEADER]: Becoming Leader. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Running, Role: LEADER
I20260926 22:44:36.794816 17312 consensus_queue.cc:237] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } }
I20260926 22:44:36.795662 16801 catalog_manager.cc:5495] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8727cc594e844889b22d525801ae1c8 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "f8727cc594e844889b22d525801ae1c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } health_report { overall_health: UNKNOWN } } }
Sep 26 22:44:36 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:36.826656 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.819452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38393 (local address 127.0.0.1:56058)
0926 22:44:36.819649 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.819660 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.819727 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.819847 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.819848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.820218 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:36.820268 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.820270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.820993 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.820994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.821520 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.821522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.821833 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.821834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.821881 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.822359 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:36.822363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:36.823925 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:36.825544 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.825551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.825553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.825739 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.825956 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:36.825957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:36.825957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:36.826017 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:36.826260 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:36.826268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:36.826309 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.826391 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.826507 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":126,"threads_started":1}
I20260926 22:44:36.826687 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.819511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.14.184.254:38393)
0926 22:44:36.819582 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.819585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.819731 (+ 146us) server_negotiation.cc:407] Connection header received
0926 22:44:36.819773 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.819773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.819782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.819834 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:36.820360 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.820923 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.821594 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.821766 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.821816 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.824014 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:36.824015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:36.824016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:36.824023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:36.825448 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.825834 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.825835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.825835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.825890 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:36.826080 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:36.826081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:36.826082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:36.826184 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:36.826255 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.826414 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.826575 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":42}
I20260926 22:44:36.828647 16801 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:56058:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 22:44:36.828877 16801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:36.830521 17078 tablet_service.cc:1467] Processing CreateTablet for tablet 39a87c0bed37488eb6be4bdb4b705a2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=004a6d2dd91048d49f87705d840c7231]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:36.830559 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 39a87c0bed37488eb6be4bdb4b705a2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=004a6d2dd91048d49f87705d840c7231]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:36.830710 17078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a87c0bed37488eb6be4bdb4b705a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.830767 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a87c0bed37488eb6be4bdb4b705a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.830755 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 39a87c0bed37488eb6be4bdb4b705a2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=004a6d2dd91048d49f87705d840c7231]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:36.830902 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a87c0bed37488eb6be4bdb4b705a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.831774 17305 tablet_bootstrap.cc:492] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: Bootstrap starting.
I20260926 22:44:36.832338 17305 tablet_bootstrap.cc:654] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.832509 17304 tablet_bootstrap.cc:492] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: Bootstrap starting.
I20260926 22:44:36.832516 17308 tablet_bootstrap.cc:492] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: Bootstrap starting.
I20260926 22:44:36.832988 17304 tablet_bootstrap.cc:654] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.833033 17308 tablet_bootstrap.cc:654] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.833158 17305 tablet_bootstrap.cc:492] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: No bootstrap required, opened a new log
I20260926 22:44:36.833201 17305 ts_tablet_manager.cc:1397] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:36.833353 17305 raft_consensus.cc:348] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.833452 17305 raft_consensus.cc:374] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.833474 17305 raft_consensus.cc:729] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04715c541de4699a33454169cb60c4a, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.833528 17305 consensus_queue.cc:260] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.833630 17305 ts_tablet_manager.cc:1428] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.833688 17309 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.833736 17304 tablet_bootstrap.cc:492] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: No bootstrap required, opened a new log
I20260926 22:44:36.833782 17309 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:36.833822 17304 ts_tablet_manager.cc:1397] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:36.833987 17304 raft_consensus.cc:348] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.834085 17304 raft_consensus.cc:374] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.834107 17304 raft_consensus.cc:729] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf210d17b5e4ba880615c98f4da3272, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.834164 17304 consensus_queue.cc:260] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.834256 17304 ts_tablet_manager.cc:1428] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.834317 17310 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.834559 17310 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:36.834638 17308 tablet_bootstrap.cc:492] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: No bootstrap required, opened a new log
I20260926 22:44:36.834733 17308 ts_tablet_manager.cc:1397] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:36.834872 17308 raft_consensus.cc:348] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.834941 17308 raft_consensus.cc:374] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.834955 17308 raft_consensus.cc:729] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.835021 17308 consensus_queue.cc:260] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.835117 17308 ts_tablet_manager.cc:1428] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.835166 17312 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.835258 17312 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 22:44:36.877604 17005 tablet.cc:2369] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:36.879853 17309 raft_consensus.cc:482] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:36.879941 17309 raft_consensus.cc:504] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.880347 17309 leader_election.cc:288] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373), 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:36.884107 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.880546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:36273 (local address 127.14.184.193:36373)
0926 22:44:36.880602 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.880604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.880812 (+ 208us) server_negotiation.cc:407] Connection header received
0926 22:44:36.880840 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.880841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.880848 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.880892 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.881035 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.881791 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.883185 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.883752 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.883836 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.883844 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.884002 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.884038 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 22:44:36.884107 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.880587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:60193 (local address 127.14.184.195:46419)
0926 22:44:36.880635 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.880637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.880645 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:36.880673 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.880674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.880683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.880742 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:36.880950 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.881733 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.883186 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.883752 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.883839 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.883845 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.884002 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.884038 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 22:44:36.884155 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.880486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:46419 (local address 127.14.184.194:60193)
0926 22:44:36.880538 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.880551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.880596 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.880773 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.880774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.880776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.880860 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.880862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.881816 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.881817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.883093 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.883096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.883838 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.883839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.883924 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.883927 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.883985 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.884038 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":33}
I20260926 22:44:36.884164 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.880491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:36373 (local address 127.14.184.194:36273)
0926 22:44:36.880742 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.880751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.880799 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.880909 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.880910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.880912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:36.880971 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.880973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.881861 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.881862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.883093 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.883096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.883861 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.883862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.883924 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.883927 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.883985 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.884038 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":65,"thread_start_us":125,"threads_started":1}
I20260926 22:44:36.884435 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a87c0bed37488eb6be4bdb4b705a2f" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272" is_pre_election: true
I20260926 22:44:36.884546 17238 raft_consensus.cc:2454] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 0.
I20260926 22:44:36.884613 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a87c0bed37488eb6be4bdb4b705a2f" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8727cc594e844889b22d525801ae1c8" is_pre_election: true
I20260926 22:44:36.884722 16958 raft_consensus.cc:2454] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 0.
I20260926 22:44:36.884752 17031 leader_election.cc:302] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf210d17b5e4ba880615c98f4da3272, e04715c541de4699a33454169cb60c4a; no voters:
I20260926 22:44:36.884872 17309 raft_consensus.cc:2781] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:36.884912 17309 raft_consensus.cc:482] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:36.884943 17309 raft_consensus.cc:3037] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.885607 17309 raft_consensus.cc:504] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.885740 17309 leader_election.cc:288] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 election: Requested vote from peers f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373), 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:36.885893 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a87c0bed37488eb6be4bdb4b705a2f" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272"
I20260926 22:44:36.885974 17238 raft_consensus.cc:3037] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.886590 17238 raft_consensus.cc:2454] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 1.
I20260926 22:44:36.886723 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a87c0bed37488eb6be4bdb4b705a2f" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8727cc594e844889b22d525801ae1c8"
I20260926 22:44:36.886747 17031 leader_election.cc:302] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf210d17b5e4ba880615c98f4da3272, e04715c541de4699a33454169cb60c4a; no voters:
I20260926 22:44:36.886821 16958 raft_consensus.cc:3037] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.886839 17309 raft_consensus.cc:2781] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:36.886988 17309 raft_consensus.cc:686] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [term 1 LEADER]: Becoming Leader. State: Replica: e04715c541de4699a33454169cb60c4a, State: Running, Role: LEADER
I20260926 22:44:36.887107 17309 consensus_queue.cc:237] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.887542 17326 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: TxnStatusTablet state changed. Reason: New leader e04715c541de4699a33454169cb60c4a. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.887610 17326 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:36.887693 16958 raft_consensus.cc:2454] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 1.
I20260926 22:44:36.887837 16801 catalog_manager.cc:5495] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a reported cstate change: term changed from 0 to 1, leader changed from <none> to e04715c541de4699a33454169cb60c4a (127.14.184.194). New cstate: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:36.888175 17328 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 22:44:36.888329 17328 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 22:44:36.895913 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.893244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.14.184.194:38953)
0926 22:44:36.893291 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.893293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.893301 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:36.893402 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.893403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.893416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.893465 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:36.893590 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.894023 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.895173 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.895400 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.895452 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.895643 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.895725 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.895781 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.895851 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.895868 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:36.895977 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.893176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.0.0.1:39846)
0926 22:44:36.893251 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.893263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.893321 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.893479 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.893480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.893482 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:36.893530 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.893533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.894092 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.894093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.895078 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.895082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.895468 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.895469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.895539 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.895556 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.895785 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.895793 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.895849 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.895899 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 22:44:36.897609 17238 raft_consensus.cc:1261] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Refusing update from remote peer e04715c541de4699a33454169cb60c4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:36.897626 16958 raft_consensus.cc:1261] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Refusing update from remote peer e04715c541de4699a33454169cb60c4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:36.897895 17326 consensus_queue.cc:1035] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:36.898015 17309 consensus_queue.cc:1035] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:36.899050 17312 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: TxnStatusTablet state changed. Reason: New leader e04715c541de4699a33454169cb60c4a. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.899142 17312 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:36.899145 17310 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: TxnStatusTablet state changed. Reason: New leader e04715c541de4699a33454169cb60c4a. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.899221 17310 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:36.900311 17310 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.900324 17326 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.900391 17310 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P 3cf210d17b5e4ba880615c98f4da3272: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:36.900400 17326 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:36.900445 17326 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.900488 17326 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P e04715c541de4699a33454169cb60c4a: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:36.900657 17312 tablet_replica.cc:439] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } } }
I20260926 22:44:36.900736 17312 tablet_replica.cc:442] T 39a87c0bed37488eb6be4bdb4b705a2f P f8727cc594e844889b22d525801ae1c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:36.901041 17332 mvcc.cc:205] Tried to move back new op lower bound from 7333735124570542080 to 7333735124530176000. Current Snapshot: MvccSnapshot[applied={T|T < 7333735124570542080 or (T in {7333735124570542080})}]
I20260926 22:44:36.901734 17333 mvcc.cc:205] Tried to move back new op lower bound from 7333735124570542080 to 7333735124530176000. Current Snapshot: MvccSnapshot[applied={T|T < 7333735124570542080}]
I20260926 22:44:36.905423 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.902856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.14.184.193:36373)
0926 22:44:36.902897 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.902898 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.902910 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:44:36.902986 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.902987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.903003 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.903067 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:36.903152 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.903898 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.904602 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.904910 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.904971 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.905104 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.905235 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.905316 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.905335 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.905355 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:44:36.910189 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.907695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:51351 (local address 127.14.184.194:38953)
0926 22:44:36.907734 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.907736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.907830 (+ 94us) server_negotiation.cc:407] Connection header received
0926 22:44:36.907891 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.907891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.907909 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.907948 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:36.908059 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.908473 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.909596 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.909764 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.909813 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.909996 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.910018 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.910054 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.910119 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.910137 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:36.910238 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.907618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.14.184.193:51351)
0926 22:44:36.907763 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.907773 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.907835 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.907960 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.907961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.907962 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:36.908006 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.908008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.908537 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.908537 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.909517 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.909519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.909831 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.909832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.909896 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.909910 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.910069 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.910070 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.910113 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.910153 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260926 22:44:36.912601 17238 raft_consensus.cc:1261] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Refusing update from remote peer f8727cc594e844889b22d525801ae1c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:36.912613 17086 raft_consensus.cc:1261] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Refusing update from remote peer f8727cc594e844889b22d525801ae1c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:36.912827 17314 consensus_queue.cc:1035] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:36.912941 17312 consensus_queue.cc:1035] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:36.921393 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.918537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:36373 (local address 127.14.184.194:50101)
0926 22:44:36.918653 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:36.918686 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:36.918737 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:36.918906 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:36.918907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:36.918908 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:36.918964 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.918965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.919837 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.919838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.920551 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:36.920554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.921056 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.921057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:36.921109 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.921121 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.921278 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.921279 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:36.921316 (+ 37us) client_negotiation.cc:240] Negotiation successful
0926 22:44:36.921346 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 22:44:36.921409 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:36.918593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:50101 (local address 127.14.184.193:36373)
0926 22:44:36.918633 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:36.918635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:36.918732 (+ 97us) server_negotiation.cc:407] Connection header received
0926 22:44:36.918810 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:36.918811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:36.918821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:36.918865 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:36.919039 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.919766 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.920676 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:36.920986 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:36.921048 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:36.921191 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:36.921223 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:36.921279 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:36.921323 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 22:44:36.921344 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:36.930421 16801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:36.930526 16801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:36.931217 16801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Sending DeleteTablet for 3 replicas of tablet 23a2fed3e3d1465793a46cb04dcb853d
I20260926 22:44:36.931569 17218 tablet_service.cc:1514] Processing DeleteTablet for tablet 23a2fed3e3d1465793a46cb04dcb853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:36 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:52130
I20260926 22:44:36.931563 17078 tablet_service.cc:1514] Processing DeleteTablet for tablet 23a2fed3e3d1465793a46cb04dcb853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:36 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39838
I20260926 22:44:36.931900 16938 tablet_service.cc:1514] Processing DeleteTablet for tablet 23a2fed3e3d1465793a46cb04dcb853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:36 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39466
I20260926 22:44:36.932123 16801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:36.932271 16801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:36.932571 17352 tablet_replica.cc:330] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: stopping tablet replica
I20260926 22:44:36.933434 17078 tablet_service.cc:1467] Processing CreateTablet for tablet 37af639064cb46648f9b13d72b29c0f3 (DEFAULT_TABLE table=test-table [id=ad692fa4fa5545a2a27483ed1a7f0800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:36.933519 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 37af639064cb46648f9b13d72b29c0f3 (DEFAULT_TABLE table=test-table [id=ad692fa4fa5545a2a27483ed1a7f0800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:36.933554 17078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37af639064cb46648f9b13d72b29c0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.933605 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37af639064cb46648f9b13d72b29c0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.934950 17305 tablet_bootstrap.cc:492] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Bootstrap starting.
I20260926 22:44:36.934968 17304 tablet_bootstrap.cc:492] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Bootstrap starting.
I20260926 22:44:36.935249 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 37af639064cb46648f9b13d72b29c0f3 (DEFAULT_TABLE table=test-table [id=ad692fa4fa5545a2a27483ed1a7f0800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:36.935372 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37af639064cb46648f9b13d72b29c0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:36.935384 17304 tablet_bootstrap.cc:654] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.935384 17305 tablet_bootstrap.cc:654] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.936058 17304 tablet_bootstrap.cc:492] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: No bootstrap required, opened a new log
I20260926 22:44:36.936107 17304 ts_tablet_manager.cc:1397] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:36.936196 17305 tablet_bootstrap.cc:492] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: No bootstrap required, opened a new log
I20260926 22:44:36.936235 17305 ts_tablet_manager.cc:1397] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:36.936235 17304 raft_consensus.cc:348] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.936295 17304 raft_consensus.cc:374] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.936316 17304 raft_consensus.cc:729] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf210d17b5e4ba880615c98f4da3272, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.936367 17304 consensus_queue.cc:260] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.936378 17305 raft_consensus.cc:348] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.936434 17305 raft_consensus.cc:374] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.936451 17305 raft_consensus.cc:729] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04715c541de4699a33454169cb60c4a, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.936450 17304 ts_tablet_manager.cc:1428] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:36.936491 17305 consensus_queue.cc:260] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.936596 17305 ts_tablet_manager.cc:1428] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:36.936935 17308 tablet_bootstrap.cc:492] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Bootstrap starting.
I20260926 22:44:36.937156 17352 raft_consensus.cc:2229] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:36.937273 17352 raft_consensus.cc:2258] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:36.937352 17308 tablet_bootstrap.cc:654] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:36.937727 17352 ts_tablet_manager.cc:1905] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:36.938802 17350 tablet_replica.cc:330] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: stopping tablet replica
I20260926 22:44:36.939040 17351 tablet_replica.cc:330] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: stopping tablet replica
I20260926 22:44:36.939191 17351 raft_consensus.cc:2229] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:36.939254 17352 ts_tablet_manager.cc:1918] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:36.939323 17352 log.cc:1196] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/23a2fed3e3d1465793a46cb04dcb853d
I20260926 22:44:36.939443 17350 raft_consensus.cc:2229] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:36.939560 17351 raft_consensus.cc:2258] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:36.939589 17352 ts_tablet_manager.cc:1939] T 23a2fed3e3d1465793a46cb04dcb853d P f8727cc594e844889b22d525801ae1c8: Deleting consensus metadata
I20260926 22:44:36.939690 17350 raft_consensus.cc:2258] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:36.939980 16786 catalog_manager.cc:4841] TS f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373): tablet 23a2fed3e3d1465793a46cb04dcb853d (table test-table [id=d4400126d3da4a21a47807affeaa30b7]) successfully deleted
I20260926 22:44:36.940232 17308 tablet_bootstrap.cc:492] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: No bootstrap required, opened a new log
I20260926 22:44:36.940279 17308 ts_tablet_manager.cc:1397] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 22:44:36.940433 17308 raft_consensus.cc:348] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.940507 17308 raft_consensus.cc:374] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:36.940534 17308 raft_consensus.cc:729] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Initialized, Role: FOLLOWER
I20260926 22:44:36.940573 17308 consensus_queue.cc:260] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.940662 17308 ts_tablet_manager.cc:1428] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:36.942965 17351 ts_tablet_manager.cc:1905] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:36.944531 17351 ts_tablet_manager.cc:1918] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:36.944612 17351 log.cc:1196] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/23a2fed3e3d1465793a46cb04dcb853d
I20260926 22:44:36.944829 17351 ts_tablet_manager.cc:1939] T 23a2fed3e3d1465793a46cb04dcb853d P e04715c541de4699a33454169cb60c4a: Deleting consensus metadata
I20260926 22:44:36.945199 16788 catalog_manager.cc:4841] TS e04715c541de4699a33454169cb60c4a (127.14.184.194:38953): tablet 23a2fed3e3d1465793a46cb04dcb853d (table test-table [id=d4400126d3da4a21a47807affeaa30b7]) successfully deleted
I20260926 22:44:36.950925 17350 ts_tablet_manager.cc:1905] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:36.952544 17350 ts_tablet_manager.cc:1918] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:36.952617 17350 log.cc:1196] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/23a2fed3e3d1465793a46cb04dcb853d
I20260926 22:44:36.952862 17350 ts_tablet_manager.cc:1939] T 23a2fed3e3d1465793a46cb04dcb853d P 3cf210d17b5e4ba880615c98f4da3272: Deleting consensus metadata
I20260926 22:44:36.953249 16787 catalog_manager.cc:4841] TS 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419): tablet 23a2fed3e3d1465793a46cb04dcb853d (table test-table [id=d4400126d3da4a21a47807affeaa30b7]) successfully deleted
I20260926 22:44:36.980031 17314 raft_consensus.cc:482] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:36.980134 17314 raft_consensus.cc:504] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.980358 17314 leader_election.cc:288] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e04715c541de4699a33454169cb60c4a (127.14.184.194:38953), 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:36.980507 17086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37af639064cb46648f9b13d72b29c0f3" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04715c541de4699a33454169cb60c4a" is_pre_election: true
I20260926 22:44:36.980597 17086 raft_consensus.cc:2454] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 0.
I20260926 22:44:36.980674 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37af639064cb46648f9b13d72b29c0f3" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272" is_pre_election: true
I20260926 22:44:36.980731 16892 leader_election.cc:302] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e04715c541de4699a33454169cb60c4a, f8727cc594e844889b22d525801ae1c8; no voters:
I20260926 22:44:36.980763 17238 raft_consensus.cc:2454] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 0.
I20260926 22:44:36.980818 17314 raft_consensus.cc:2781] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:36.980845 17314 raft_consensus.cc:482] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:36.980865 17314 raft_consensus.cc:3037] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.981752 17314 raft_consensus.cc:504] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.981889 17314 leader_election.cc:288] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 election: Requested vote from peers e04715c541de4699a33454169cb60c4a (127.14.184.194:38953), 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:36.982065 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37af639064cb46648f9b13d72b29c0f3" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272"
I20260926 22:44:36.982141 17238 raft_consensus.cc:3037] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.982110 17086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37af639064cb46648f9b13d72b29c0f3" candidate_uuid: "f8727cc594e844889b22d525801ae1c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04715c541de4699a33454169cb60c4a"
I20260926 22:44:36.982175 17086 raft_consensus.cc:3037] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:36.982928 17086 raft_consensus.cc:2454] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 1.
I20260926 22:44:36.983017 17238 raft_consensus.cc:2454] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8727cc594e844889b22d525801ae1c8 in term 1.
I20260926 22:44:36.983083 16892 leader_election.cc:302] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e04715c541de4699a33454169cb60c4a, f8727cc594e844889b22d525801ae1c8; no voters:
I20260926 22:44:36.983156 17314 raft_consensus.cc:2781] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:36.983192 17314 raft_consensus.cc:686] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 1 LEADER]: Becoming Leader. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Running, Role: LEADER
I20260926 22:44:36.983240 17314 consensus_queue.cc:237] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:36.983773 16801 catalog_manager.cc:5495] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8727cc594e844889b22d525801ae1c8 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "f8727cc594e844889b22d525801ae1c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:36.999105 17237 raft_consensus.cc:1261] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Refusing update from remote peer f8727cc594e844889b22d525801ae1c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:36.999426 17086 raft_consensus.cc:1261] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Refusing update from remote peer f8727cc594e844889b22d525801ae1c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:36.999449 17312 consensus_queue.cc:1035] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:36.999616 17312 consensus_queue.cc:1035] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:37.001120 17331 mvcc.cc:205] Tried to move back new op lower bound from 7333735124986863616 to 7333735124923781120. Current Snapshot: MvccSnapshot[applied={T|T < 7333735124986863616}]
W20260926 22:44:37.006134 17332 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 22:44:38.526700 16801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:38.526810 16801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:38.527595 16801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Sending DeleteTablet for 3 replicas of tablet 37af639064cb46648f9b13d72b29c0f3
I20260926 22:44:38.527916 16938 tablet_service.cc:1514] Processing DeleteTablet for tablet 37af639064cb46648f9b13d72b29c0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:38 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39466
I20260926 22:44:38.527921 17218 tablet_service.cc:1514] Processing DeleteTablet for tablet 37af639064cb46648f9b13d72b29c0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:38 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:52130
I20260926 22:44:38.529176 17379 tablet_replica.cc:330] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: stopping tablet replica
I20260926 22:44:38.529247 17379 raft_consensus.cc:2229] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:38.529292 17379 raft_consensus.cc:2258] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:38.529558 17379 ts_tablet_manager.cc:1905] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:38.530359 17078 tablet_service.cc:1514] Processing DeleteTablet for tablet 37af639064cb46648f9b13d72b29c0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:38 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39838
I20260926 22:44:38.530546 17381 tablet_replica.cc:330] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: stopping tablet replica
I20260926 22:44:38.530610 17381 raft_consensus.cc:2229] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:38.530653 17381 raft_consensus.cc:2258] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:38.530740 17378 tablet_replica.cc:330] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: stopping tablet replica
I20260926 22:44:38.530805 17378 raft_consensus.cc:2229] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:38.530869 17378 raft_consensus.cc:2258] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:38.530894 17381 ts_tablet_manager.cc:1905] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:38.531113 17378 ts_tablet_manager.cc:1905] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:38.531423 17379 ts_tablet_manager.cc:1918] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 22:44:38.531533 17379 log.cc:1196] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/37af639064cb46648f9b13d72b29c0f3
I20260926 22:44:38.531805 17379 ts_tablet_manager.cc:1939] T 37af639064cb46648f9b13d72b29c0f3 P 3cf210d17b5e4ba880615c98f4da3272: Deleting consensus metadata
I20260926 22:44:38.532166 16787 catalog_manager.cc:4841] TS 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419): tablet 37af639064cb46648f9b13d72b29c0f3 (table test-table [id=ad692fa4fa5545a2a27483ed1a7f0800]) successfully deleted
I20260926 22:44:38.532727 17378 ts_tablet_manager.cc:1918] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 22:44:38.532786 17378 log.cc:1196] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/37af639064cb46648f9b13d72b29c0f3
I20260926 22:44:38.532935 17381 ts_tablet_manager.cc:1918] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 22:44:38.533001 17381 log.cc:1196] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/37af639064cb46648f9b13d72b29c0f3
I20260926 22:44:38.533048 17378 ts_tablet_manager.cc:1939] T 37af639064cb46648f9b13d72b29c0f3 P f8727cc594e844889b22d525801ae1c8: Deleting consensus metadata
I20260926 22:44:38.533269 17381 ts_tablet_manager.cc:1939] T 37af639064cb46648f9b13d72b29c0f3 P e04715c541de4699a33454169cb60c4a: Deleting consensus metadata
I20260926 22:44:38.533373 16786 catalog_manager.cc:4841] TS f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373): tablet 37af639064cb46648f9b13d72b29c0f3 (table test-table [id=ad692fa4fa5545a2a27483ed1a7f0800]) successfully deleted
I20260926 22:44:38.533565 16788 catalog_manager.cc:4841] TS e04715c541de4699a33454169cb60c4a (127.14.184.194:38953): tablet 37af639064cb46648f9b13d72b29c0f3 (table test-table [id=ad692fa4fa5545a2a27483ed1a7f0800]) successfully deleted
Sep 26 22:44:38 dist-test-slave-3986 krb5kdc[16756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462678, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 22:44:38 dist-test-slave-3986 krb5kdc[16756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462678, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:38.551151 17388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.544583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.14.184.254:38393)
0926 22:44:38.544713 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.544715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.544724 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:38.544772 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.544773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.544782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.544846 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:38.545353 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.545845 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.546568 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.546802 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.546855 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.548952 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:38.548953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:38.548953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:38.548960 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:38.550212 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:38.550513 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:38.550514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:38.550514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:38.550567 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:38.550760 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:38.550761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:38.550761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:38.550862 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:38.550950 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.551022 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.551083 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260926 22:44:38.552523 16801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:38.552676 16801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:38.553952 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 1c188ea8e0924c549f67caf1bd2c2cbf (DEFAULT_TABLE table=test-table [id=7e4e544bc00746be8159e03a066d6300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:38.554020 17078 tablet_service.cc:1467] Processing CreateTablet for tablet 1c188ea8e0924c549f67caf1bd2c2cbf (DEFAULT_TABLE table=test-table [id=7e4e544bc00746be8159e03a066d6300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:38.554117 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c188ea8e0924c549f67caf1bd2c2cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:38.554178 17078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c188ea8e0924c549f67caf1bd2c2cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:38.554842 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 1c188ea8e0924c549f67caf1bd2c2cbf (DEFAULT_TABLE table=test-table [id=7e4e544bc00746be8159e03a066d6300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:38.554965 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c188ea8e0924c549f67caf1bd2c2cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:38.555436 17389 tablet_bootstrap.cc:492] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Bootstrap starting.
I20260926 22:44:38.555805 17389 tablet_bootstrap.cc:654] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:38.556370 17392 tablet_bootstrap.cc:492] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Bootstrap starting.
I20260926 22:44:38.556392 17389 tablet_bootstrap.cc:492] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: No bootstrap required, opened a new log
I20260926 22:44:38.556437 17389 ts_tablet_manager.cc:1397] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:38.556572 17390 tablet_bootstrap.cc:492] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Bootstrap starting.
I20260926 22:44:38.556546 17389 raft_consensus.cc:348] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.556634 17389 raft_consensus.cc:374] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:38.556659 17389 raft_consensus.cc:729] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Initialized, Role: FOLLOWER
I20260926 22:44:38.556733 17389 consensus_queue.cc:260] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.556777 17392 tablet_bootstrap.cc:654] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:38.556834 17389 ts_tablet_manager.cc:1428] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:38.556927 17390 tablet_bootstrap.cc:654] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:38.557538 17390 tablet_bootstrap.cc:492] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: No bootstrap required, opened a new log
I20260926 22:44:38.557593 17390 ts_tablet_manager.cc:1397] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:38.557690 17392 tablet_bootstrap.cc:492] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: No bootstrap required, opened a new log
I20260926 22:44:38.557736 17392 ts_tablet_manager.cc:1397] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:38.557730 17390 raft_consensus.cc:348] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.557793 17390 raft_consensus.cc:374] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:38.557816 17390 raft_consensus.cc:729] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04715c541de4699a33454169cb60c4a, State: Initialized, Role: FOLLOWER
I20260926 22:44:38.557840 17392 raft_consensus.cc:348] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.557880 17390 consensus_queue.cc:260] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.557910 17392 raft_consensus.cc:374] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:38.557937 17392 raft_consensus.cc:729] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf210d17b5e4ba880615c98f4da3272, State: Initialized, Role: FOLLOWER
I20260926 22:44:38.557991 17392 consensus_queue.cc:260] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.558142 17390 ts_tablet_manager.cc:1428] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 22:44:38.558161 17392 ts_tablet_manager.cc:1428] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:38.617266 17326 raft_consensus.cc:482] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:38.617359 17326 raft_consensus.cc:504] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.617550 17326 leader_election.cc:288] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373), 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:38.617796 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c188ea8e0924c549f67caf1bd2c2cbf" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8727cc594e844889b22d525801ae1c8" is_pre_election: true
I20260926 22:44:38.617805 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c188ea8e0924c549f67caf1bd2c2cbf" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272" is_pre_election: true
I20260926 22:44:38.617913 17238 raft_consensus.cc:2454] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 0.
I20260926 22:44:38.617913 16958 raft_consensus.cc:2454] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 0.
I20260926 22:44:38.618067 17031 leader_election.cc:302] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf210d17b5e4ba880615c98f4da3272, e04715c541de4699a33454169cb60c4a; no voters:
I20260926 22:44:38.618177 17326 raft_consensus.cc:2781] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:38.618242 17326 raft_consensus.cc:482] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:38.618270 17326 raft_consensus.cc:3037] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:38.619055 17326 raft_consensus.cc:504] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.619208 17326 leader_election.cc:288] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 election: Requested vote from peers f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373), 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419)
I20260926 22:44:38.619362 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c188ea8e0924c549f67caf1bd2c2cbf" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8727cc594e844889b22d525801ae1c8"
I20260926 22:44:38.619378 17238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c188ea8e0924c549f67caf1bd2c2cbf" candidate_uuid: "e04715c541de4699a33454169cb60c4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf210d17b5e4ba880615c98f4da3272"
I20260926 22:44:38.619458 16958 raft_consensus.cc:3037] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:38.619467 17238 raft_consensus.cc:3037] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:38.620191 17238 raft_consensus.cc:2454] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 1.
I20260926 22:44:38.620191 16958 raft_consensus.cc:2454] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e04715c541de4699a33454169cb60c4a in term 1.
I20260926 22:44:38.620386 17031 leader_election.cc:302] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf210d17b5e4ba880615c98f4da3272, e04715c541de4699a33454169cb60c4a; no voters:
I20260926 22:44:38.620489 17326 raft_consensus.cc:2781] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:38.620559 17326 raft_consensus.cc:686] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 1 LEADER]: Becoming Leader. State: Replica: e04715c541de4699a33454169cb60c4a, State: Running, Role: LEADER
I20260926 22:44:38.620611 17326 consensus_queue.cc:237] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.621150 16800 catalog_manager.cc:5495] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a reported cstate change: term changed from 0 to 1, leader changed from <none> to e04715c541de4699a33454169cb60c4a (127.14.184.194). New cstate: current_term: 1 leader_uuid: "e04715c541de4699a33454169cb60c4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:38.632321 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.630187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.14.184.194:38953)
0926 22:44:38.630361 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.630362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.630369 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:38.630386 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.630387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.630395 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.630451 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:38.630547 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.631093 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.631790 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.631975 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.632023 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.632131 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.632155 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.632195 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.632225 (+ 30us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.632240 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":121,"threads_started":1}
I20260926 22:44:38.636404 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.634203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:49633 (local address 127.14.184.194:38953)
0926 22:44:38.634256 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.634258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.634348 (+ 90us) server_negotiation.cc:407] Connection header received
0926 22:44:38.634418 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.634419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.634428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.634478 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:38.634577 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.635081 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.635760 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.635960 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.636036 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.636170 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.636196 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.636247 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.636303 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.636329 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":32}
I20260926 22:44:38.636456 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.634141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.14.184.194:49633)
0926 22:44:38.634281 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:38.634291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:38.634362 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:38.634481 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:38.634481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:38.634483 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:38.634521 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.634523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.635151 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.635152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.635684 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.635685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.636039 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.636039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.636088 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.636091 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.636249 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.636250 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:38.636300 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 22:44:38.636347 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260926 22:44:38.638193 16958 raft_consensus.cc:1261] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Refusing update from remote peer e04715c541de4699a33454169cb60c4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:38.638226 17238 raft_consensus.cc:1261] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Refusing update from remote peer e04715c541de4699a33454169cb60c4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:38.638381 17326 consensus_queue.cc:1035] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:38.638487 17361 consensus_queue.cc:1035] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:38.648242 16800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:38.648345 16800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:38.648968 16800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Sending DeleteTablet for 3 replicas of tablet 1c188ea8e0924c549f67caf1bd2c2cbf
I20260926 22:44:38.649260 16938 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c188ea8e0924c549f67caf1bd2c2cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:38 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39466
I20260926 22:44:38.649305 17078 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c188ea8e0924c549f67caf1bd2c2cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:38 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39838
I20260926 22:44:38.649360 17218 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c188ea8e0924c549f67caf1bd2c2cbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:38 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:52130
I20260926 22:44:38.649384 17381 tablet_replica.cc:330] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: stopping tablet replica
I20260926 22:44:38.649435 17381 raft_consensus.cc:2229] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:38.649446 17378 tablet_replica.cc:330] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: stopping tablet replica
I20260926 22:44:38.649443 17379 tablet_replica.cc:330] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: stopping tablet replica
I20260926 22:44:38.649490 17378 raft_consensus.cc:2229] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:38.649489 17379 raft_consensus.cc:2229] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:38.649516 17381 raft_consensus.cc:2258] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:38.649528 17378 raft_consensus.cc:2258] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:38.649530 17379 raft_consensus.cc:2258] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:38.649796 17381 ts_tablet_manager.cc:1905] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:38.649804 16800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:38.649925 16800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:38.650975 17378 ts_tablet_manager.cc:1905] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:38.651228 17218 tablet_service.cc:1467] Processing CreateTablet for tablet 89a8d425150c4e0b8e100628d7d4c4ee (DEFAULT_TABLE table=test-table [id=ee7a32bcfe7041c0bf6e71de549989b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:38.651333 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a8d425150c4e0b8e100628d7d4c4ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:38.651592 17381 ts_tablet_manager.cc:1918] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:38.651635 17381 log.cc:1196] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/1c188ea8e0924c549f67caf1bd2c2cbf
I20260926 22:44:38.651849 17381 ts_tablet_manager.cc:1939] T 1c188ea8e0924c549f67caf1bd2c2cbf P e04715c541de4699a33454169cb60c4a: Deleting consensus metadata
I20260926 22:44:38.652468 17378 ts_tablet_manager.cc:1918] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:38.652516 17378 log.cc:1196] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/1c188ea8e0924c549f67caf1bd2c2cbf
I20260926 22:44:38.652724 17378 ts_tablet_manager.cc:1939] T 1c188ea8e0924c549f67caf1bd2c2cbf P f8727cc594e844889b22d525801ae1c8: Deleting consensus metadata
I20260926 22:44:38.652997 17379 ts_tablet_manager.cc:1905] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:38.653470 17392 tablet_bootstrap.cc:492] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: Bootstrap starting.
I20260926 22:44:38.653849 17392 tablet_bootstrap.cc:654] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:38.654400 17379 ts_tablet_manager.cc:1918] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:38.654448 17379 log.cc:1196] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/1c188ea8e0924c549f67caf1bd2c2cbf
I20260926 22:44:38.654470 17392 tablet_bootstrap.cc:492] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: No bootstrap required, opened a new log
I20260926 22:44:38.654510 17392 ts_tablet_manager.cc:1397] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:38.654652 17379 ts_tablet_manager.cc:1939] T 1c188ea8e0924c549f67caf1bd2c2cbf P 3cf210d17b5e4ba880615c98f4da3272: Deleting consensus metadata
I20260926 22:44:38.654697 17392 raft_consensus.cc:348] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.654772 17392 raft_consensus.cc:374] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:38.654794 17392 raft_consensus.cc:729] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf210d17b5e4ba880615c98f4da3272, State: Initialized, Role: FOLLOWER
I20260926 22:44:38.654840 17392 consensus_queue.cc:260] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.654870 16786 catalog_manager.cc:4841] TS f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373): tablet 1c188ea8e0924c549f67caf1bd2c2cbf (table test-table [id=7e4e544bc00746be8159e03a066d6300]) successfully deleted
I20260926 22:44:38.654953 17392 ts_tablet_manager.cc:1428] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:38.655059 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 89a8d425150c4e0b8e100628d7d4c4ee (DEFAULT_TABLE table=test-table [id=ee7a32bcfe7041c0bf6e71de549989b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:38.655153 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a8d425150c4e0b8e100628d7d4c4ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:38.655434 16787 catalog_manager.cc:4841] TS 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195:46419): tablet 1c188ea8e0924c549f67caf1bd2c2cbf (table test-table [id=7e4e544bc00746be8159e03a066d6300]) successfully deleted
I20260926 22:44:38.656306 17389 tablet_bootstrap.cc:492] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8: Bootstrap starting.
I20260926 22:44:38.656581 16788 catalog_manager.cc:4841] TS e04715c541de4699a33454169cb60c4a (127.14.184.194:38953): tablet 1c188ea8e0924c549f67caf1bd2c2cbf (table test-table [id=7e4e544bc00746be8159e03a066d6300]) successfully deleted
I20260926 22:44:38.656644 17389 tablet_bootstrap.cc:654] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:38.656894 17078 tablet_service.cc:1467] Processing CreateTablet for tablet 89a8d425150c4e0b8e100628d7d4c4ee (DEFAULT_TABLE table=test-table [id=ee7a32bcfe7041c0bf6e71de549989b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:38.656994 17078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a8d425150c4e0b8e100628d7d4c4ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:38.658349 17390 tablet_bootstrap.cc:492] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Bootstrap starting.
I20260926 22:44:38.658552 17389 tablet_bootstrap.cc:492] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8: No bootstrap required, opened a new log
I20260926 22:44:38.658602 17389 ts_tablet_manager.cc:1397] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:38.658727 17389 raft_consensus.cc:348] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.658797 17389 raft_consensus.cc:374] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:38.658816 17389 raft_consensus.cc:729] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8727cc594e844889b22d525801ae1c8, State: Initialized, Role: FOLLOWER
I20260926 22:44:38.658859 17389 consensus_queue.cc:260] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.658942 17389 ts_tablet_manager.cc:1428] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:38.658974 17390 tablet_bootstrap.cc:654] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:38.659613 17390 tablet_bootstrap.cc:492] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: No bootstrap required, opened a new log
I20260926 22:44:38.659662 17390 ts_tablet_manager.cc:1397] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:38.659750 17390 raft_consensus.cc:348] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.659806 17390 raft_consensus.cc:374] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:38.659824 17390 raft_consensus.cc:729] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04715c541de4699a33454169cb60c4a, State: Initialized, Role: FOLLOWER
I20260926 22:44:38.659860 17390 consensus_queue.cc:260] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.659945 17390 ts_tablet_manager.cc:1428] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260926 22:44:38.675102 17285 tablet.cc:2369] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:38.677172 17395 raft_consensus.cc:482] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:38.677232 17395 raft_consensus.cc:504] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.677560 17395 leader_election.cc:288] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373), e04715c541de4699a33454169cb60c4a (127.14.184.194:38953)
I20260926 22:44:38.681245 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.677870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:44381 (local address 127.14.184.194:38953)
0926 22:44:38.677919 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.677921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.677930 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:38.677977 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.677977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.677986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.678040 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:38.678181 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.679001 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.680435 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.680867 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.680945 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.680953 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.681137 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.681175 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:38.681245 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.677710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.14.184.195:44381)
0926 22:44:38.677872 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:38.677882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:38.677923 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:38.678051 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:38.678052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:38.678055 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:38.678120 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.678122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.679079 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.679080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.680353 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.680357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.680942 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.680943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.681044 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.681050 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 22:44:38.681125 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:44:38.681167 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260926 22:44:38.681473 17098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a8d425150c4e0b8e100628d7d4c4ee" candidate_uuid: "3cf210d17b5e4ba880615c98f4da3272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04715c541de4699a33454169cb60c4a" is_pre_election: true
I20260926 22:44:38.681578 17098 raft_consensus.cc:2454] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cf210d17b5e4ba880615c98f4da3272 in term 0.
I20260926 22:44:38.681790 17172 leader_election.cc:302] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf210d17b5e4ba880615c98f4da3272, e04715c541de4699a33454169cb60c4a; no voters:
I20260926 22:44:38.681910 17395 raft_consensus.cc:2781] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:38.681952 17395 raft_consensus.cc:482] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:38.681970 17395 raft_consensus.cc:3037] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:38.682587 17395 raft_consensus.cc:504] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.682732 17395 leader_election.cc:288] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [CANDIDATE]: Term 1 election: Requested vote from peers f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373), e04715c541de4699a33454169cb60c4a (127.14.184.194:38953)
I20260926 22:44:38.682885 17098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a8d425150c4e0b8e100628d7d4c4ee" candidate_uuid: "3cf210d17b5e4ba880615c98f4da3272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04715c541de4699a33454169cb60c4a"
I20260926 22:44:38.682969 17098 raft_consensus.cc:3037] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:38.683621 17098 raft_consensus.cc:2454] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cf210d17b5e4ba880615c98f4da3272 in term 1.
I20260926 22:44:38.683696 17411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.677752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41823 (local address 127.14.184.193:36373)
0926 22:44:38.678000 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.678002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.678009 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:38.678033 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.678035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.678046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.678110 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:38.678265 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.679010 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.682733 (+ 3723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.683350 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.683429 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.683437 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.683585 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.683626 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":70,"threads_started":1}
I20260926 22:44:38.683766 17172 leader_election.cc:302] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf210d17b5e4ba880615c98f4da3272, e04715c541de4699a33454169cb60c4a; no voters:
I20260926 22:44:38.683845 17395 raft_consensus.cc:2781] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:38.683923 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.677700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:36373 (local address 127.14.184.195:41823)
0926 22:44:38.677816 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:38.677828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:38.677887 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:38.678154 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:38.678155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:38.678157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:38.678203 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.678205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.680445 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.680446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.681822 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.681825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.683436 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.683437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.683502 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.683506 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:38.683572 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 22:44:38.683616 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260926 22:44:38.683996 17395 raft_consensus.cc:686] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 1 LEADER]: Becoming Leader. State: Replica: 3cf210d17b5e4ba880615c98f4da3272, State: Running, Role: LEADER
I20260926 22:44:38.684065 17395 consensus_queue.cc:237] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } }
I20260926 22:44:38.684202 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a8d425150c4e0b8e100628d7d4c4ee" candidate_uuid: "3cf210d17b5e4ba880615c98f4da3272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8727cc594e844889b22d525801ae1c8"
I20260926 22:44:38.684283 16957 raft_consensus.cc:3037] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:38.684378 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a8d425150c4e0b8e100628d7d4c4ee" candidate_uuid: "3cf210d17b5e4ba880615c98f4da3272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8727cc594e844889b22d525801ae1c8" is_pre_election: true
I20260926 22:44:38.684666 16800 catalog_manager.cc:5495] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cf210d17b5e4ba880615c98f4da3272 (127.14.184.195). New cstate: current_term: 1 leader_uuid: "3cf210d17b5e4ba880615c98f4da3272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cf210d17b5e4ba880615c98f4da3272" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 46419 } health_report { overall_health: HEALTHY } } }
I20260926 22:44:38.685119 16957 raft_consensus.cc:2454] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cf210d17b5e4ba880615c98f4da3272 in term 1.
I20260926 22:44:38.692747 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.690303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.14.184.195:46419)
0926 22:44:38.690430 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.690431 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.690438 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:38.690472 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.690473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.690489 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.690574 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:38.690690 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.691178 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.691953 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.692171 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.692244 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.692405 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.692508 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.692587 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.692625 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.692644 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":87,"threads_started":1}
I20260926 22:44:38.697274 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.695103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41405 (local address 127.14.184.194:38953)
0926 22:44:38.695146 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.695148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.695156 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:38.695223 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.695224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.695234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.695287 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:38.695391 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.695827 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.696582 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.696786 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.696850 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.697032 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.697063 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.697125 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.697186 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.697205 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:44:38.697311 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.694974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38953 (local address 127.14.184.195:41405)
0926 22:44:38.695103 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:38.695113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:38.695164 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:38.695290 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:38.695291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:38.695292 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:38.695344 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.695345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.695901 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.695902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.696484 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.696486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.696861 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.696862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.696933 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.696945 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.697127 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.697129 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:44:38.697181 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:44:38.697218 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":77,"threads_started":1}
I20260926 22:44:38.699277 16957 raft_consensus.cc:1261] T 89a8d425150c4e0b8e100628d7d4c4ee P f8727cc594e844889b22d525801ae1c8 [term 1 FOLLOWER]: Refusing update from remote peer 3cf210d17b5e4ba880615c98f4da3272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:38.699276 17098 raft_consensus.cc:1261] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Refusing update from remote peer 3cf210d17b5e4ba880615c98f4da3272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:38.699487 17395 consensus_queue.cc:1035] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8727cc594e844889b22d525801ae1c8" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:38.699584 17412 consensus_queue.cc:1035] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e04715c541de4699a33454169cb60c4a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:38.707166 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.704357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:46419 (local address 127.14.184.194:37013)
0926 22:44:38.704396 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:38.704408 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:38.704445 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:38.704585 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:38.704586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:38.704587 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:38.704622 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.704624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.705479 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.705480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.706369 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:38.706371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.706860 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.706861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:38.706906 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.706911 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.707057 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.707058 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:38.707087 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 22:44:38.707120 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 22:44:38.707173 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:38.704405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:37013 (local address 127.14.184.195:46419)
0926 22:44:38.704439 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:38.704441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:38.704449 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:38.704500 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:38.704500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:38.704507 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:38.704566 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:38.704692 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.705408 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.706482 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:38.706798 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:38.706862 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:38.706974 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:38.707005 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:38.707057 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:38.707090 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 22:44:38.707108 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:44:38.707473 17363 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 22:44:40.224996 16801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:40.225111 16801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:40.225881 16801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b95f529dd034b6b9e265a89efab2ac4: Sending DeleteTablet for 3 replicas of tablet 89a8d425150c4e0b8e100628d7d4c4ee
I20260926 22:44:40.226152 17077 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a8d425150c4e0b8e100628d7d4c4ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:40 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39838
I20260926 22:44:40.226186 17218 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a8d425150c4e0b8e100628d7d4c4ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:40 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:52130
I20260926 22:44:40.226334 16938 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a8d425150c4e0b8e100628d7d4c4ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:40 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39466
I20260926 22:44:40.226337 17445 tablet_replica.cc:330] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: stopping tablet replica
I20260926 22:44:40.226410 17445 raft_consensus.cc:2229] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:40.226450 17445 raft_consensus.cc:2258] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:40.226632 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 16873
I20260926 22:44:40.228735 17445 ts_tablet_manager.cc:1905] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:40.230458 17445 ts_tablet_manager.cc:1918] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 22:44:40.230509 17445 log.cc:1196] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/89a8d425150c4e0b8e100628d7d4c4ee
I20260926 22:44:40.230769 17445 ts_tablet_manager.cc:1939] T 89a8d425150c4e0b8e100628d7d4c4ee P e04715c541de4699a33454169cb60c4a: Deleting consensus metadata
I20260926 22:44:40.231076 16788 catalog_manager.cc:4841] TS e04715c541de4699a33454169cb60c4a (127.14.184.194:38953): tablet 89a8d425150c4e0b8e100628d7d4c4ee (table test-table [id=ee7a32bcfe7041c0bf6e71de549989b3]) successfully deleted
W20260926 22:44:40.232383 16786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:36373 (error 108)
I20260926 22:44:40.232611 17446 tablet_replica.cc:330] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: stopping tablet replica
I20260926 22:44:40.232684 17446 raft_consensus.cc:2229] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:40.232767 17446 raft_consensus.cc:2258] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:40.232918 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17013
I20260926 22:44:40.233069 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:40.232752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:36373 (local address 127.0.0.1:57788)
0926 22:44:40.232960 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:40.233012 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:36373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
W20260926 22:44:40.234516 16786 catalog_manager.cc:4584] TS f8727cc594e844889b22d525801ae1c8 (127.14.184.193:36373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89a8d425150c4e0b8e100628d7d4c4ee: Network error: Client connection negotiation failed: client connection to 127.14.184.193:36373: connect: Connection refused (error 111)
I20260926 22:44:40.238646 17446 ts_tablet_manager.cc:1905] T 89a8d425150c4e0b8e100628d7d4c4ee P 3cf210d17b5e4ba880615c98f4da3272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:40.238983 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17153
I20260926 22:44:40.243472 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 16769
I20260926 22:44:40.243489 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:40.243377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:46419 (local address 127.0.0.1:51496)
0926 22:44:40.243424 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:40.243454 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.195:46419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
2026-09-26T22:44:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5297 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:40 dist-test-slave-3986 krb5kdc[17452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:40 dist-test-slave-3986 krb5kdc[17452](info): set up 2 sockets
Sep 26 22:44:40 dist-test-slave-3986 krb5kdc[17452](info): commencing operation
krb5kdc: starting...
W20260926 22:44:41.274559 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:41.383354 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33319
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:33319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:41.461865 17465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:41.462052 17465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:41.462074 17465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:41.463214 17465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:41.463253 17465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:41.463266 17465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:41.463315 17465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:41.463331 17465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:41.464699 17465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:33319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:41.464927 17465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:41.465159 17465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:41.469767 17471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:41.469780 17473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.469795 17465 server_base.cc:1034] running on GCE node
W20260926 22:44:41.469796 17470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.470301 17465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:41.470583 17465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:41.471737 17465 hybrid_clock.cc:648] HybridClock initialized: now 1790462681471725 us; error 36 us; skew 500 ppm
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:41.474035 17465 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:41.474468 17465 webserver.cc:460] Webserver started at http://127.14.184.254:39783/ using document root <none> and password file <none>
I20260926 22:44:41.474725 17465 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:41.474797 17465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:41.474948 17465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:41.475927 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "3e395265c0cc408a8a64399f8d675848"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "15685708b8d2b9ff74fcaee8ea3b2ab3"
server_key_iv: "e3238cd7fef9604b72e07fcfd0d9cbc5"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.476271 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "3e395265c0cc408a8a64399f8d675848"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "15685708b8d2b9ff74fcaee8ea3b2ab3"
server_key_iv: "e3238cd7fef9604b72e07fcfd0d9cbc5"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.477509 17465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:41.478178 17480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.478346 17465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:41.478440 17465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "3e395265c0cc408a8a64399f8d675848"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "15685708b8d2b9ff74fcaee8ea3b2ab3"
server_key_iv: "e3238cd7fef9604b72e07fcfd0d9cbc5"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.478520 17465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:41.488366 17465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:41.490101 17465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:41.490257 17465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:41.494050 17465 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:33319
I20260926 22:44:41.494079 17532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:33319 every 8 connection(s)
I20260926 22:44:41.494421 17465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:41.495265 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:41.497078 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17465
I20260926 22:44:41.497176 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:41.497344 15075 external_mini_cluster.cc:1428] Setting key 3f427d2292f893d55ed684c2c0110099
I20260926 22:44:41.497593 17533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848: Bootstrap starting.
I20260926 22:44:41.498260 17533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:41.498564 17533 log.cc:824] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:41.499346 17533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848: No bootstrap required, opened a new log
I20260926 22:44:41.501552 17533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e395265c0cc408a8a64399f8d675848" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33319 } }
I20260926 22:44:41.501729 17533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:41.501765 17533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e395265c0cc408a8a64399f8d675848, State: Initialized, Role: FOLLOWER
I20260926 22:44:41.501883 17533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e395265c0cc408a8a64399f8d675848" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33319 } }
I20260926 22:44:41.501951 17533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:41.501989 17533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:41.502035 17533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:41.502781 17533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e395265c0cc408a8a64399f8d675848" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33319 } }
I20260926 22:44:41.502892 17533 leader_election.cc:302] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e395265c0cc408a8a64399f8d675848; no voters:
I20260926 22:44:41.503067 17533 leader_election.cc:288] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:41.503095 17538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:41.503201 17538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [term 1 LEADER]: Becoming Leader. State: Replica: 3e395265c0cc408a8a64399f8d675848, State: Running, Role: LEADER
I20260926 22:44:41.503333 17533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:41.503311 17538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e395265c0cc408a8a64399f8d675848" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33319 } }
I20260926 22:44:41.503625 17538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e395265c0cc408a8a64399f8d675848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e395265c0cc408a8a64399f8d675848" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33319 } } }
I20260926 22:44:41.503695 17538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:41.503815 17538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e395265c0cc408a8a64399f8d675848. Latest consensus state: current_term: 1 leader_uuid: "3e395265c0cc408a8a64399f8d675848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e395265c0cc408a8a64399f8d675848" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33319 } } }
I20260926 22:44:41.503899 17538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:41.504122 17543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:41.504609 17543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:41.507578 17543 catalog_manager.cc:1347] Generated new cluster ID: 340b74e331504d779bc81104361d2721
I20260926 22:44:41.507629 17543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:41.508852 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.497847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.14.184.254:33319)
0926 22:44:41.498016 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:41.498025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:41.498057 (+ 32us) server_negotiation.cc:407] Connection header received
0926 22:44:41.498536 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:41.498542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:41.498589 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:41.498762 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:41.499525 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.500887 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.501641 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.502262 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.502382 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.505320 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:41.505324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:41.505324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:41.505335 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:41.507327 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:41.507747 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:41.507750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:41.507751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:41.507839 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:41.508060 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:41.508061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:41.508061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:41.508284 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:41.508461 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:41.508528 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 22:44:41.508661 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260926 22:44:41.513909 17543 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:41.514560 17543 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:41.524138 17543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e395265c0cc408a8a64399f8d675848: Generated new TSK 0
I20260926 22:44:41.524339 17543 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:41.556893 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33319
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:41.657912 17559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:41.658124 17559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:41.658159 17559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:41.659310 17559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:41.659365 17559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:41.659409 17559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:41.660813 17559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:41.661054 17559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:41.661259 17559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:41.666031 17567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:41.666014 17564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.666143 17559 server_base.cc:1034] running on GCE node
W20260926 22:44:41.666015 17565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.666505 17559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:41.666842 17559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:41.668027 17559 hybrid_clock.cc:648] HybridClock initialized: now 1790462681668009 us; error 33 us; skew 500 ppm
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:41.670956 17559 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:41.671331 17559 webserver.cc:460] Webserver started at http://127.14.184.193:38219/ using document root <none> and password file <none>
I20260926 22:44:41.671500 17559 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:41.671537 17559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:41.671622 17559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:41.672551 17559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "599dd3843d8e4c639ec478ccb3275dc4"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "41f51e8efdf5b9752acda5f15281e2a3"
server_key_iv: "1cd69c93bfc3939e2ebdba14bb6794ca"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.672833 17559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "599dd3843d8e4c639ec478ccb3275dc4"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "41f51e8efdf5b9752acda5f15281e2a3"
server_key_iv: "1cd69c93bfc3939e2ebdba14bb6794ca"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.673986 17559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:41.674578 17574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.674788 17559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:41.674889 17559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "599dd3843d8e4c639ec478ccb3275dc4"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "41f51e8efdf5b9752acda5f15281e2a3"
server_key_iv: "1cd69c93bfc3939e2ebdba14bb6794ca"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.674971 17559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:41.688959 17559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:41.690635 17559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:41.690806 17559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:41.691023 17559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:41.691313 17559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:41.691371 17559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.691411 17559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:41.691439 17559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.696959 17559 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:42263
I20260926 22:44:41.697021 17687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:42263 every 8 connection(s)
I20260926 22:44:41.697325 17559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:41.701958 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17559
I20260926 22:44:41.702059 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:41.702255 15075 external_mini_cluster.cc:1428] Setting key 6bdf34a4d7df935f00e78fdb78abc889
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:41.706743 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.697988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59225 (local address 127.14.184.254:33319)
0926 22:44:41.698048 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:41.698050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:41.698236 (+ 186us) server_negotiation.cc:407] Connection header received
0926 22:44:41.698778 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:41.698779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:41.698789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:41.698861 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:41.699763 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.700279 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.701209 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.701403 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.701453 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.703764 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:41.703766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:41.703766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:41.703774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:41.705393 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:41.705783 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:41.705784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:41.705785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:41.705847 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:41.706083 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:41.706084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:41.706085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:41.706199 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:41.706272 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:41.706532 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 22:44:41.706619 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:44:41.706919 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.697953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33319 (local address 127.14.184.193:59225)
0926 22:44:41.698111 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:41.698152 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:41.698630 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:41.698915 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:41.698919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:41.699507 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:41.699685 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:41.699690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.700353 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.700354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:41.701138 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:41.701140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.701468 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.701469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:41.701589 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.702134 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:41.702144 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:41.703667 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:41.705480 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:41.705486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:41.705488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:41.705710 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:41.705912 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:41.705913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:41.705914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:41.706019 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:41.706263 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:41.706269 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:41.706404 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 22:44:41.706508 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 22:44:41.706663 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260926 22:44:41.707397 17688 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33319
I20260926 22:44:41.707506 17688 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:41.707707 17688 heartbeater.cc:511] Master 127.14.184.254:33319 requested a full tablet report, sending...
I20260926 22:44:41.708218 17497 ts_manager.cc:194] Registered new tserver with Master: 599dd3843d8e4c639ec478ccb3275dc4 (127.14.184.193:42263)
I20260926 22:44:41.708817 17497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:59225
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:41.727344 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33319
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:41.808100 17693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:41.808300 17693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:41.808321 17693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:41.809397 17693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:41.809433 17693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:41.809458 17693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:41.810796 17693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:41.810959 17693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:41.811120 17693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:41.816072 17701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:41.816103 17698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:41.816112 17699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.816433 17693 server_base.cc:1034] running on GCE node
I20260926 22:44:41.816663 17693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:41.816903 17693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:41.818065 17693 hybrid_clock.cc:648] HybridClock initialized: now 1790462681818058 us; error 44 us; skew 500 ppm
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:41.820535 17693 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:41.821024 17693 webserver.cc:460] Webserver started at http://127.14.184.194:43813/ using document root <none> and password file <none>
I20260926 22:44:41.821283 17693 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:41.821350 17693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:41.821498 17693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:41.822546 17693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "deaf9041e5514fe0b554648529ea8a5b"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "18a6ab0d95868c55bd9e856bad6e3095"
server_key_iv: "443511b1ed376363983a17d4255ee8f1"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.822961 17693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "deaf9041e5514fe0b554648529ea8a5b"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "18a6ab0d95868c55bd9e856bad6e3095"
server_key_iv: "443511b1ed376363983a17d4255ee8f1"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.824347 17693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:41.825111 17708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.825271 17693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:41.825369 17693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "deaf9041e5514fe0b554648529ea8a5b"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "18a6ab0d95868c55bd9e856bad6e3095"
server_key_iv: "443511b1ed376363983a17d4255ee8f1"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.825448 17693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:41.835706 17693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:41.837514 17693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:41.837667 17693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:41.837903 17693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:41.838208 17693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:41.838258 17693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.838299 17693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:41.838325 17693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.844121 17693 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:38521
I20260926 22:44:41.844168 17821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:38521 every 8 connection(s)
I20260926 22:44:41.844461 17693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:41.851910 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17693
I20260926 22:44:41.852022 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:41.852213 15075 external_mini_cluster.cc:1428] Setting key 328c8127bfaca67f97b4af4187441abf
I20260926 22:44:41.853300 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.845106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33319 (local address 127.14.184.194:37399)
0926 22:44:41.845273 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:41.845313 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:41.845778 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:41.846017 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:41.846020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:41.846549 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:41.846807 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:41.846812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.847438 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.847439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:41.848227 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:41.848228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.848558 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.848559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:41.848650 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.849075 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:41.849082 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:41.850302 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:41.851909 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:41.851916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:41.851918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:41.852209 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:41.852405 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:41.852406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:41.852407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:41.852527 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:41.852780 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:41.852787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:41.852940 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 22:44:41.853032 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 22:44:41.853156 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260926 22:44:41.853303 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.845127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:37399 (local address 127.14.184.254:33319)
0926 22:44:41.845186 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:41.845188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:41.845402 (+ 214us) server_negotiation.cc:407] Connection header received
0926 22:44:41.845895 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:41.845896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:41.845906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:41.845966 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:41.846886 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.847361 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.848298 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.848488 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.848545 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.850396 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:41.850398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:41.850398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:41.850404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:41.851781 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:41.852282 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:41.852283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:41.852283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:41.852340 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:41.852583 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:41.852584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:41.852585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:41.852695 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:41.852783 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:41.853074 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 22:44:41.853172 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 22:44:41.853817 17822 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33319
I20260926 22:44:41.853919 17822 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:41.854082 17822 heartbeater.cc:511] Master 127.14.184.254:33319 requested a full tablet report, sending...
I20260926 22:44:41.854477 17497 ts_manager.cc:194] Registered new tserver with Master: deaf9041e5514fe0b554648529ea8a5b (127.14.184.194:38521)
I20260926 22:44:41.854913 17497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:37399
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:41.876792 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33319
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:41.954970 17827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:41.955178 17827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:41.955212 17827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:41.956307 17827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:41.956369 17827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:41.956413 17827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:41.957803 17827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:41.958039 17827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:41.958246 17827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:41.963039 17832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:41.963019 17835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.963048 17827 server_base.cc:1034] running on GCE node
W20260926 22:44:41.963029 17833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:41.963527 17827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:41.963778 17827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:41.964931 17827 hybrid_clock.cc:648] HybridClock initialized: now 1790462681964922 us; error 34 us; skew 500 ppm
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:41.967374 17827 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:41.967775 17827 webserver.cc:460] Webserver started at http://127.14.184.195:45299/ using document root <none> and password file <none>
I20260926 22:44:41.967994 17827 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:41.968056 17827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:41.968196 17827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:41.969228 17827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "307e5caaab404c3dbed4beda8ee99c16"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "760bcc5e6fa890a4a902732118f64fed"
server_key_iv: "6f5d685b8974248928239fffd8554930"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.969585 17827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "307e5caaab404c3dbed4beda8ee99c16"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "760bcc5e6fa890a4a902732118f64fed"
server_key_iv: "6f5d685b8974248928239fffd8554930"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.970961 17827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:44:41.971668 17842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.971853 17827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:41.971952 17827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "307e5caaab404c3dbed4beda8ee99c16"
format_stamp: "Formatted at 2026-09-26 22:44:41 on dist-test-slave-3986"
server_key: "760bcc5e6fa890a4a902732118f64fed"
server_key_iv: "6f5d685b8974248928239fffd8554930"
server_key_version: "encryptionkey@0"
I20260926 22:44:41.972028 17827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:41.983675 17827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:41.985460 17827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:41.985608 17827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:41.985831 17827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:41.986117 17827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:41.986166 17827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.986207 17827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:41.986233 17827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:41.992138 17827 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:37745
I20260926 22:44:41.992192 17955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:37745 every 8 connection(s)
I20260926 22:44:41.992527 17827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
Sep 26 22:44:41 dist-test-slave-3986 krb5kdc[17452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462681, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:42.001014 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17827
I20260926 22:44:42.001086 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.993232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:34867 (local address 127.14.184.254:33319)
0926 22:44:41.993309 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:41.993312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:41.993520 (+ 208us) server_negotiation.cc:407] Connection header received
0926 22:44:41.993979 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:41.993980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:41.993989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:41.994052 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:41.995080 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.995572 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.996489 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.996672 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.996728 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.998665 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:41.998708 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:41.998709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:41.998720 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:41.999907 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:42.000252 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:42.000252 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:42.000253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:42.000301 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:42.000510 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:42.000510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:42.000510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:42.000604 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:42.000653 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:42.000892 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 22:44:42.000976 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:44:42.001121 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:42.001143 17958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:41.993166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33319 (local address 127.14.184.195:34867)
0926 22:44:41.993387 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:41.993429 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:41.993901 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:41.994107 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:41.994110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:41.994803 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:41.995002 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:41.995007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.995648 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.995648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:41.996412 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:41.996413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:41.996742 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:41.996742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:41.996837 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:41.997276 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:41.997284 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:41.998585 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:41.999972 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:41.999978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:41.999979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:42.000189 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:42.000356 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:42.000357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:42.000357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:42.000460 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:42.000654 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:42.000658 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:42.000761 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 22:44:42.000840 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 22:44:42.000953 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260926 22:44:42.001335 15075 external_mini_cluster.cc:1428] Setting key 5c21e6744582ba8e8328590b32dc65c7
I20260926 22:44:42.001660 17956 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33319
I20260926 22:44:42.001754 17956 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:42.001946 17956 heartbeater.cc:511] Master 127.14.184.254:33319 requested a full tablet report, sending...
I20260926 22:44:42.002317 17497 ts_manager.cc:194] Registered new tserver with Master: 307e5caaab404c3dbed4beda8ee99c16 (127.14.184.195:37745)
I20260926 22:44:42.002707 17497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:34867
I20260926 22:44:42.013043 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 22:44:42.023592 17536 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:42.022643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.14.184.254:33319)
0926 22:44:42.022719 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:42.022721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:42.022835 (+ 114us) server_negotiation.cc:407] Connection header received
0926 22:44:42.022899 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:42.022900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:42.022909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:42.022949 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:42.023481 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58082: BlockingRecv error: recv got EOF from 127.0.0.1:58082 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 22:44:42.024461 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17559
I20260926 22:44:42.029052 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17693
I20260926 22:44:42.033121 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17827
I20260926 22:44:42.037225 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17465
2026-09-26T22:44:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1792 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:42 dist-test-slave-3986 krb5kdc[17968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:42 dist-test-slave-3986 krb5kdc[17968](info): set up 2 sockets
Sep 26 22:44:42 dist-test-slave-3986 krb5kdc[17968](info): commencing operation
krb5kdc: starting...
W20260926 22:44:43.070644 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:43 dist-test-slave-3986 krb5kdc[17968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:43.181035 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44849
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:44849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:43.259737 17981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:43.259931 17981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:43.259953 17981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:43.261004 17981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:43.261036 17981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:43.261051 17981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:43.261063 17981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:43.261075 17981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:43.262605 17981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:44849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:43.262840 17981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:43.263018 17981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:43.267611 17986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:43.267627 17987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:43.267592 17989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:43.267863 17981 server_base.cc:1034] running on GCE node
I20260926 22:44:43.268081 17981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:43.268358 17981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:43.269497 17981 hybrid_clock.cc:648] HybridClock initialized: now 1790462683269486 us; error 29 us; skew 500 ppm
Sep 26 22:44:43 dist-test-slave-3986 krb5kdc[17968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462683, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:43.271955 17981 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:43.272385 17981 webserver.cc:460] Webserver started at http://127.14.184.254:37697/ using document root <none> and password file <none>
I20260926 22:44:43.272616 17981 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:43.272679 17981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:43.272823 17981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:43.273834 17981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "9676ea25683149ecbda1561e3290764b"
format_stamp: "Formatted at 2026-09-26 22:44:43 on dist-test-slave-3986"
server_key: "d6681598a812370a593fe606890774f6"
server_key_iv: "c5bb08b4fd6bce0fe231ad66e52af1d8"
server_key_version: "encryptionkey@0"
I20260926 22:44:43.274178 17981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "9676ea25683149ecbda1561e3290764b"
format_stamp: "Formatted at 2026-09-26 22:44:43 on dist-test-slave-3986"
server_key: "d6681598a812370a593fe606890774f6"
server_key_iv: "c5bb08b4fd6bce0fe231ad66e52af1d8"
server_key_version: "encryptionkey@0"
I20260926 22:44:43.275516 17981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:44:43.276172 17996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:43.276327 17981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:43.276423 17981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "9676ea25683149ecbda1561e3290764b"
format_stamp: "Formatted at 2026-09-26 22:44:43 on dist-test-slave-3986"
server_key: "d6681598a812370a593fe606890774f6"
server_key_iv: "c5bb08b4fd6bce0fe231ad66e52af1d8"
server_key_version: "encryptionkey@0"
I20260926 22:44:43.276489 17981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:43.286048 17981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:43.288215 17981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:43.288370 17981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:43.292155 17981 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:44849
I20260926 22:44:43.292197 18048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:44849 every 8 connection(s)
I20260926 22:44:43.292667 17981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:43.293304 18049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:43.295482 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 17981
I20260926 22:44:43.295581 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:43.295646 18049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b: Bootstrap starting.
I20260926 22:44:43.295758 15075 external_mini_cluster.cc:1428] Setting key fc423fb282381d207315cc2ca32d5edc
I20260926 22:44:43.296373 18049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:43.296684 18049 log.cc:824] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:43.297302 18049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b: No bootstrap required, opened a new log
I20260926 22:44:43.299153 18049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676ea25683149ecbda1561e3290764b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44849 } }
I20260926 22:44:43.299275 18049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:43.299296 18049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9676ea25683149ecbda1561e3290764b, State: Initialized, Role: FOLLOWER
I20260926 22:44:43.299425 18049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676ea25683149ecbda1561e3290764b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44849 } }
I20260926 22:44:43.299499 18049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:43.299521 18049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:43.299583 18049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:43.300089 18049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676ea25683149ecbda1561e3290764b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44849 } }
I20260926 22:44:43.300203 18049 leader_election.cc:302] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9676ea25683149ecbda1561e3290764b; no voters:
I20260926 22:44:43.300360 18049 leader_election.cc:288] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:43.300405 18054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:43.300603 18049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:43.301505 18054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [term 1 LEADER]: Becoming Leader. State: Replica: 9676ea25683149ecbda1561e3290764b, State: Running, Role: LEADER
I20260926 22:44:43.301608 18054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676ea25683149ecbda1561e3290764b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44849 } }
I20260926 22:44:43.301942 18061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9676ea25683149ecbda1561e3290764b. Latest consensus state: current_term: 1 leader_uuid: "9676ea25683149ecbda1561e3290764b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676ea25683149ecbda1561e3290764b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44849 } } }
I20260926 22:44:43.302021 18061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [sys.catalog]: This master's current role is: LEADER
Sep 26 22:44:43 dist-test-slave-3986 krb5kdc[17968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:43.302245 18055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9676ea25683149ecbda1561e3290764b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676ea25683149ecbda1561e3290764b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44849 } } }
I20260926 22:44:43.302326 18055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:43.302239 18068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:43.303155 18068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:43.306048 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:43.296212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.14.184.254:44849)
0926 22:44:43.296368 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:43.296376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:43.296406 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:44:43.296910 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:43.296917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:43.296962 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:43.297104 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:43.297768 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:43.298617 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:43.299487 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:43.299975 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:43.300085 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:43.302732 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:43.302736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:43.302736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:43.302746 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:43.304583 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:43.304986 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:43.304988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:43.304989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:43.305066 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:43.305269 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:43.305269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:43.305270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:43.305474 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:43.305583 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:43.305725 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 22:44:43.305816 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260926 22:44:43.307013 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:44:43.307024 18068 catalog_manager.cc:1347] Generated new cluster ID: ffc80efe17d04d1bb8acd00209dd63a1
I20260926 22:44:43.307067 18068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:43.318389 18068 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:43.318957 18068 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:43.331146 18068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9676ea25683149ecbda1561e3290764b: Generated new TSK 0
I20260926 22:44:43.331331 18068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:44:43.336117 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:43.332317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.14.184.254:44849)
0926 22:44:43.332377 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:43.332380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:43.332482 (+ 102us) server_negotiation.cc:407] Connection header received
0926 22:44:43.332526 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:43.332527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:43.332538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:43.332611 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:43.332950 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:43.333775 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:43.334574 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:43.334945 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:43.335018 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:43.335659 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:43.335660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:43.335660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:43.335878 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:43.335897 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:43.335904 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:43.335959 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:43.336013 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 22:44:43.336039 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:44:43.338387 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 17981
2026-09-26T22:44:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1299 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:43 dist-test-slave-3986 krb5kdc[18082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:43 dist-test-slave-3986 krb5kdc[18082](info): set up 2 sockets
Sep 26 22:44:43 dist-test-slave-3986 krb5kdc[18082](info): commencing operation
krb5kdc: starting...
W20260926 22:44:44.358876 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:44.469408 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45789
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:45789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:44.547048 18095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:44.547278 18095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:44.547314 18095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:44.548445 18095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:44.548508 18095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:44.548533 18095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:44.548553 18095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:44.548581 18095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:44.550009 18095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:45789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:44.550273 18095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:44.550490 18095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:44.555105 18100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:44.555132 18101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:44.555204 18095 server_base.cc:1034] running on GCE node
W20260926 22:44:44.555115 18103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:44.555670 18095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:44.555955 18095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:44.557125 18095 hybrid_clock.cc:648] HybridClock initialized: now 1790462684557111 us; error 31 us; skew 500 ppm
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:44.559589 18095 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:44.560005 18095 webserver.cc:460] Webserver started at http://127.14.184.254:46417/ using document root <none> and password file <none>
I20260926 22:44:44.560220 18095 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:44.560284 18095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:44.560431 18095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:44.561468 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "e62e963aa74abdb879b9e3f351c2bb7a"
server_key_iv: "242951adc04c8109939cb633bda1ec37"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.561829 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "e62e963aa74abdb879b9e3f351c2bb7a"
server_key_iv: "242951adc04c8109939cb633bda1ec37"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.563202 18095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:44.563895 18110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.564049 18095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:44.564142 18095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "e62e963aa74abdb879b9e3f351c2bb7a"
server_key_iv: "242951adc04c8109939cb633bda1ec37"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.564217 18095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:44.588533 18095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:44.590974 18095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:44.591130 18095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:44.594940 18095 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:45789
I20260926 22:44:44.594982 18162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:45789 every 8 connection(s)
I20260926 22:44:44.595353 18095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:44.596306 18163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:44.598557 18163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad: Bootstrap starting.
I20260926 22:44:44.599145 18163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:44.599435 18163 log.cc:824] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:44.600114 18163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad: No bootstrap required, opened a new log
I20260926 22:44:44.601637 18163 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45789 } }
I20260926 22:44:44.601769 18163 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:44.601791 18163 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a1b1ac577d4d9b9935d01e9e8527ad, State: Initialized, Role: FOLLOWER
I20260926 22:44:44.601970 18163 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45789 } }
I20260926 22:44:44.602051 18163 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:44.602099 18163 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:44.602150 18163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:44.602936 18163 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45789 } }
I20260926 22:44:44.603034 18163 leader_election.cc:302] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6a1b1ac577d4d9b9935d01e9e8527ad; no voters:
I20260926 22:44:44.603192 18163 leader_election.cc:288] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:44.603261 18166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:44.603421 18166 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [term 1 LEADER]: Becoming Leader. State: Replica: c6a1b1ac577d4d9b9935d01e9e8527ad, State: Running, Role: LEADER
I20260926 22:44:44.603438 18163 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:44.603507 18166 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45789 } }
I20260926 22:44:44.603711 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18095
I20260926 22:44:44.603807 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:44.604029 15075 external_mini_cluster.cc:1428] Setting key cc04bc108d6097925393c9d97be89150
I20260926 22:44:44.604470 18167 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45789 } } }
I20260926 22:44:44.604576 18167 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:44.604720 18168 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a1b1ac577d4d9b9935d01e9e8527ad. Latest consensus state: current_term: 1 leader_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1b1ac577d4d9b9935d01e9e8527ad" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45789 } } }
I20260926 22:44:44.604789 18168 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:44.604916 18180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:44.605489 18180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:44.607049 18180 catalog_manager.cc:1347] Generated new cluster ID: b0f5708f10334f6f974ce210ca9e5c2a
I20260926 22:44:44.607105 18180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:44.613888 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:44.604771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.14.184.254:45789)
0926 22:44:44.605144 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:44.605153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:44.605192 (+ 39us) server_negotiation.cc:407] Connection header received
0926 22:44:44.605282 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:44.605287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:44.605328 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:44.605463 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:44.606101 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.607339 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.608101 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.608544 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.608658 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:44.610874 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:44.610883 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:44.610883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:44.610893 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:44.612439 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:44.612792 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:44.612795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:44.612795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:44.612890 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:44.613087 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:44.613088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:44.613088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:44.613308 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:44.613465 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:44.613559 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 22:44:44.613711 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":75,"threads_started":1}
I20260926 22:44:44.620687 18180 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:44.621362 18180 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:44.623927 18180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6a1b1ac577d4d9b9935d01e9e8527ad: Generated new TSK 0
I20260926 22:44:44.624076 18180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:44.651751 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45789
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:44.732857 18189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:44.733059 18189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:44.733083 18189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:44.734179 18189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:44.734230 18189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:44.734256 18189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:44.735785 18189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:44.735961 18189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:44.736133 18189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:44.740861 18194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:44.740886 18197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:44.740934 18189 server_base.cc:1034] running on GCE node
W20260926 22:44:44.740895 18195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:44.741305 18189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:44.741572 18189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:44.742719 18189 hybrid_clock.cc:648] HybridClock initialized: now 1790462684742711 us; error 33 us; skew 500 ppm
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:44.745199 18189 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:44.745687 18189 webserver.cc:460] Webserver started at http://127.14.184.193:39533/ using document root <none> and password file <none>
I20260926 22:44:44.745898 18189 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:44.745958 18189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:44.746086 18189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:44.747117 18189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "da5157c50c8b41979419e9308fbd55e5"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "8516db50662620885d5aa54d73beb595"
server_key_iv: "2aba7ca91e1668ad2d0b64843a8359b2"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.747463 18189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "da5157c50c8b41979419e9308fbd55e5"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "8516db50662620885d5aa54d73beb595"
server_key_iv: "2aba7ca91e1668ad2d0b64843a8359b2"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.748646 18189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:44.749281 18204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.749428 18189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:44.749531 18189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "da5157c50c8b41979419e9308fbd55e5"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "8516db50662620885d5aa54d73beb595"
server_key_iv: "2aba7ca91e1668ad2d0b64843a8359b2"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.749603 18189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:44.761057 18189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:44.763211 18189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:44.763361 18189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:44.763566 18189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:44.763859 18189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:44.763918 18189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.763953 18189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:44.763979 18189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.769862 18189 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:40593
I20260926 22:44:44.769909 18317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:40593 every 8 connection(s)
I20260926 22:44:44.770210 18189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:44.776141 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18189
I20260926 22:44:44.776252 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:44.776449 15075 external_mini_cluster.cc:1428] Setting key af3cf17a4c0c0aa277708f6759949fbf
I20260926 22:44:44.779368 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:44.770890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:52833 (local address 127.14.184.254:45789)
0926 22:44:44.770960 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:44.770962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:44.771156 (+ 194us) server_negotiation.cc:407] Connection header received
0926 22:44:44.771697 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:44.771698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:44.771711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:44.771770 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:44.772697 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.773206 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.774187 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.774402 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.774485 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:44.776551 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:44.776552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:44.776553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:44.776560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:44.777978 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:44.778391 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:44.778392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:44.778393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:44.778454 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:44.778714 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:44.778716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:44.778716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:44.778850 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:44.778918 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:44.779178 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 22:44:44.779259 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:44:44.779613 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:44.770865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45789 (local address 127.14.184.193:52833)
0926 22:44:44.771038 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:44.771073 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:44.771588 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:44.771844 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:44.771848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:44.772410 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:44.772595 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:44.772600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.773299 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.773300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:44.774100 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:44.774101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.774496 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.774497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:44.774615 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:44.775061 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:44.775070 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:44.776450 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:44.778073 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:44.778080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:44.778082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:44.778310 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:44.778520 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:44.778520 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:44.778521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:44.778635 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:44.778919 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:44.778926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:44.779055 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 22:44:44.779157 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 22:44:44.779354 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260926 22:44:44.780040 18318 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45789
I20260926 22:44:44.780143 18318 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:44.780326 18318 heartbeater.cc:511] Master 127.14.184.254:45789 requested a full tablet report, sending...
I20260926 22:44:44.780814 18127 ts_manager.cc:194] Registered new tserver with Master: da5157c50c8b41979419e9308fbd55e5 (127.14.184.193:40593)
I20260926 22:44:44.781416 18127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:52833
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:44.800937 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45789
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:44.880435 18323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:44.880625 18323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:44.880648 18323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:44.881700 18323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:44.881733 18323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:44.881759 18323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:44.883172 18323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:44.883350 18323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:44.883526 18323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:44.888231 18328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:44.888223 18331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:44.888223 18329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:44.888480 18323 server_base.cc:1034] running on GCE node
I20260926 22:44:44.888758 18323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:44.889060 18323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:44.890269 18323 hybrid_clock.cc:648] HybridClock initialized: now 1790462684890243 us; error 36 us; skew 500 ppm
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:44.892792 18323 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:44.893198 18323 webserver.cc:460] Webserver started at http://127.14.184.194:44269/ using document root <none> and password file <none>
I20260926 22:44:44.893428 18323 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:44.893492 18323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:44.893632 18323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:44.894608 18323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "a19e3cfad7d74779bbfbd620ae3bc793"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "09646635457c00e0c2fe280252e2bee2"
server_key_iv: "eec6cf2c38da83ca12bacc24d537d82b"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.895016 18323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "a19e3cfad7d74779bbfbd620ae3bc793"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "09646635457c00e0c2fe280252e2bee2"
server_key_iv: "eec6cf2c38da83ca12bacc24d537d82b"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.896260 18323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:44.896924 18338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.897087 18323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:44.897179 18323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "a19e3cfad7d74779bbfbd620ae3bc793"
format_stamp: "Formatted at 2026-09-26 22:44:44 on dist-test-slave-3986"
server_key: "09646635457c00e0c2fe280252e2bee2"
server_key_iv: "eec6cf2c38da83ca12bacc24d537d82b"
server_key_version: "encryptionkey@0"
I20260926 22:44:44.897256 18323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:44.908157 18323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:44.909756 18323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:44.909909 18323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:44.910105 18323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:44.910375 18323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:44.910423 18323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.910455 18323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:44.910480 18323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:44.916159 18323 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:37833
I20260926 22:44:44.916211 18451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:37833 every 8 connection(s)
I20260926 22:44:44.916508 18323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:44:44 dist-test-slave-3986 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462684, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:44.924749 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:44.917130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:52745 (local address 127.14.184.254:45789)
0926 22:44:44.917187 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:44.917189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:44.917381 (+ 192us) server_negotiation.cc:407] Connection header received
0926 22:44:44.917856 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:44.917857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:44.917870 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:44.917923 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:44.918775 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.919275 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.920195 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.920387 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.920479 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:44.922446 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:44.922447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:44.922448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:44.922454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:44.923595 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:44.923969 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:44.923969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:44.923970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:44.924018 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:44.924277 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:44.924277 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:44.924277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:44.924378 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:44.924426 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:44.924632 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 22:44:44.924690 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:44:44.924937 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:44.917072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45789 (local address 127.14.184.194:52745)
0926 22:44:44.917211 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:44.917248 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:44.917752 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:44.917982 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:44.917986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:44.918504 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:44.918683 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:44.918687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.919367 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.919368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:44.920110 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:44.920112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:44.920484 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:44.920485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:44.920599 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:44.921077 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:44.921084 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:44.922367 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:44.923678 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:44.923691 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:44.923692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:44.923905 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:44.924096 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:44.924096 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:44.924097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:44.924222 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:44.924429 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:44.924433 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:44.924533 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 22:44:44.924610 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 22:44:44.924716 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260926 22:44:44.925374 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18323
I20260926 22:44:44.925407 18452 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45789
I20260926 22:44:44.925488 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:44.925493 18452 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:44.925643 18452 heartbeater.cc:511] Master 127.14.184.254:45789 requested a full tablet report, sending...
I20260926 22:44:44.925681 15075 external_mini_cluster.cc:1428] Setting key 234e4c1f6f562acae8d4022878c894c8
I20260926 22:44:44.926009 18127 ts_manager.cc:194] Registered new tserver with Master: a19e3cfad7d74779bbfbd620ae3bc793 (127.14.184.194:37833)
I20260926 22:44:44.926499 18127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:52745
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:44.947563 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45789
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:45.027011 18457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:45.027207 18457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:45.027231 18457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:45.028357 18457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:45.028398 18457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:45.028450 18457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:45.029901 18457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:45.030134 18457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:45.030360 18457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:45.035233 18465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:45.035231 18462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:45.035254 18463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:45.035439 18457 server_base.cc:1034] running on GCE node
I20260926 22:44:45.035609 18457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:45.035871 18457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:45.037025 18457 hybrid_clock.cc:648] HybridClock initialized: now 1790462685036996 us; error 44 us; skew 500 ppm
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462685, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:45.039469 18457 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:45.039927 18457 webserver.cc:460] Webserver started at http://127.14.184.195:46821/ using document root <none> and password file <none>
I20260926 22:44:45.040156 18457 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:45.040220 18457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:45.040359 18457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:45.041378 18457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "31aa8802a3a3458da8007721c8f61334"
format_stamp: "Formatted at 2026-09-26 22:44:45 on dist-test-slave-3986"
server_key: "fa454b0dbf546bd29ea9324f374aadd6"
server_key_iv: "0b03bb5fc06307845c2843f7e3ea7fbe"
server_key_version: "encryptionkey@0"
I20260926 22:44:45.041754 18457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "31aa8802a3a3458da8007721c8f61334"
format_stamp: "Formatted at 2026-09-26 22:44:45 on dist-test-slave-3986"
server_key: "fa454b0dbf546bd29ea9324f374aadd6"
server_key_iv: "0b03bb5fc06307845c2843f7e3ea7fbe"
server_key_version: "encryptionkey@0"
I20260926 22:44:45.043135 18457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 22:44:45.043804 18472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:45.043959 18457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:45.044050 18457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "31aa8802a3a3458da8007721c8f61334"
format_stamp: "Formatted at 2026-09-26 22:44:45 on dist-test-slave-3986"
server_key: "fa454b0dbf546bd29ea9324f374aadd6"
server_key_iv: "0b03bb5fc06307845c2843f7e3ea7fbe"
server_key_version: "encryptionkey@0"
I20260926 22:44:45.044123 18457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:45.059659 18457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:45.061347 18457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:45.061501 18457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:45.061735 18457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:45.062048 18457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:45.062112 18457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:45.062147 18457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:45.062175 18457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:45.068114 18457 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33809
I20260926 22:44:45.068148 18585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33809 every 8 connection(s)
I20260926 22:44:45.068470 18457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:45.071915 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18457
I20260926 22:44:45.072019 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:45.072217 15075 external_mini_cluster.cc:1428] Setting key d06f6127957e41f8b48318651d6087fc
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462685, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:45.077890 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:45.069149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:59597 (local address 127.14.184.254:45789)
0926 22:44:45.069191 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:45.069193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:45.069429 (+ 236us) server_negotiation.cc:407] Connection header received
0926 22:44:45.069945 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:45.069946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:45.069958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:45.070024 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:45.071001 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:45.071499 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:45.072968 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:45.073222 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:45.073300 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:45.075387 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:45.075389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:45.075389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:45.075396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:45.076571 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:45.076961 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:45.076962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:45.076962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:45.077021 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:45.077269 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:45.077270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:45.077270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:45.077389 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:45.077455 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:45.077715 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 22:44:45.077802 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:45.078085 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:45.069123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45789 (local address 127.14.184.195:59597)
0926 22:44:45.069298 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:45.069349 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:45.069836 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:45.070095 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:45.070099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:45.070654 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:45.070903 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:45.070907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:45.071592 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:45.071593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:45.072878 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:45.072881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:45.073310 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:45.073311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:45.073448 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:45.073856 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:45.073868 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:45.075276 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:45.076654 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:45.076661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:45.076663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:45.076886 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:45.077094 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:45.077095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:45.077095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:45.077200 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:45.077457 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:45.077464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:45.077593 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 22:44:45.077692 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 22:44:45.077852 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260926 22:44:45.078529 18586 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45789
I20260926 22:44:45.078614 18586 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:45.078804 18586 heartbeater.cc:511] Master 127.14.184.254:45789 requested a full tablet report, sending...
I20260926 22:44:45.079147 18127 ts_manager.cc:194] Registered new tserver with Master: 31aa8802a3a3458da8007721c8f61334 (127.14.184.195:33809)
I20260926 22:44:45.079512 18127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:59597
I20260926 22:44:45.083859 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462685, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462685, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:45.105036 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:45.098292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.14.184.254:45789)
0926 22:44:45.098355 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:45.098358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:45.098471 (+ 113us) server_negotiation.cc:407] Connection header received
0926 22:44:45.098516 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:45.098516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:45.098525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:45.098574 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:45.099153 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:45.099627 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:45.100442 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:45.100647 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:45.100708 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:45.102841 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:45.102842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:45.102842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:45.102849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:45.103944 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:45.104277 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:45.104277 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:45.104278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:45.104323 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:45.104468 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:45.104469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:45.104469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:45.104588 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:45.104658 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:45.104801 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 22:44:45.104886 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260926 22:44:45.105254 18127 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53926
I20260926 22:44:45.105700 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18189
I20260926 22:44:45.110168 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18323
I20260926 22:44:45.114892 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18457
I20260926 22:44:45.119266 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18095
2026-09-26T22:44:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1782 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18598](info): set up 2 sockets
Sep 26 22:44:45 dist-test-slave-3986 krb5kdc[18598](info): commencing operation
krb5kdc: starting...
W20260926 22:44:46.150759 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:46.260725 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37743
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:37743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:46.338326 18611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:46.338533 18611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:46.338555 18611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:46.339722 18611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:46.339761 18611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:46.339775 18611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:46.339788 18611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:46.339799 18611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:46.341337 18611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:37743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:46.341511 18611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:46.341660 18611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:46.346138 18619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:46.346141 18616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:46.346233 18611 server_base.cc:1034] running on GCE node
W20260926 22:44:46.346170 18617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:46.346580 18611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:46.346890 18611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:46.348028 18611 hybrid_clock.cc:648] HybridClock initialized: now 1790462686348015 us; error 29 us; skew 500 ppm
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:46.353401 18611 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:46.353802 18611 webserver.cc:460] Webserver started at http://127.14.184.254:34875/ using document root <none> and password file <none>
I20260926 22:44:46.354038 18611 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:46.354101 18611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:46.354274 18611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:46.355334 18611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "b17cf7abb44842928a3c958683bc3d58"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "4b9ab68ca8e7cf2cf015b0e6bd642d2d"
server_key_iv: "17cd90369e87f6bde23d95af7479a928"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.355674 18611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "b17cf7abb44842928a3c958683bc3d58"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "4b9ab68ca8e7cf2cf015b0e6bd642d2d"
server_key_iv: "17cd90369e87f6bde23d95af7479a928"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.356920 18611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:46.357580 18626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.357731 18611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:46.357825 18611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "b17cf7abb44842928a3c958683bc3d58"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "4b9ab68ca8e7cf2cf015b0e6bd642d2d"
server_key_iv: "17cd90369e87f6bde23d95af7479a928"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.357903 18611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:46.367097 18611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:46.368834 18611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:46.368996 18611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:46.372712 18611 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:37743
I20260926 22:44:46.372756 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:37743 every 8 connection(s)
I20260926 22:44:46.373260 18611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:46.373886 18679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:46.374612 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18611
I20260926 22:44:46.374739 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:46.374902 15075 external_mini_cluster.cc:1428] Setting key 61b09ca682cde506da3f9acc974e0707
I20260926 22:44:46.376892 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58: Bootstrap starting.
I20260926 22:44:46.377579 18679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:46.377885 18679 log.cc:824] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:46.378558 18679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58: No bootstrap required, opened a new log
I20260926 22:44:46.380810 18679 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17cf7abb44842928a3c958683bc3d58" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37743 } }
I20260926 22:44:46.380946 18679 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:46.380968 18679 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17cf7abb44842928a3c958683bc3d58, State: Initialized, Role: FOLLOWER
I20260926 22:44:46.381101 18679 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17cf7abb44842928a3c958683bc3d58" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37743 } }
I20260926 22:44:46.381175 18679 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:46.381206 18679 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:46.381253 18679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:46.381799 18679 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17cf7abb44842928a3c958683bc3d58" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37743 } }
I20260926 22:44:46.381914 18679 leader_election.cc:302] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17cf7abb44842928a3c958683bc3d58; no voters:
I20260926 22:44:46.382078 18679 leader_election.cc:288] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:46.382107 18684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:46.382254 18684 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [term 1 LEADER]: Becoming Leader. State: Replica: b17cf7abb44842928a3c958683bc3d58, State: Running, Role: LEADER
I20260926 22:44:46.382383 18679 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:46.382756 18684 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17cf7abb44842928a3c958683bc3d58" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37743 } }
I20260926 22:44:46.383632 18685 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17cf7abb44842928a3c958683bc3d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17cf7abb44842928a3c958683bc3d58" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37743 } } }
I20260926 22:44:46.383728 18685 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:46.383869 18685 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17cf7abb44842928a3c958683bc3d58. Latest consensus state: current_term: 1 leader_uuid: "b17cf7abb44842928a3c958683bc3d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17cf7abb44842928a3c958683bc3d58" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37743 } } }
I20260926 22:44:46.383925 18685 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:46.384236 18697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:46.385211 18697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:46.385687 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.375411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.14.184.254:37743)
0926 22:44:46.375594 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:46.375603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:46.375638 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:44:46.376142 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:46.376151 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:46.376214 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:46.376373 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:46.376933 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.378288 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.379098 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.379721 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.379847 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.382123 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:46.382127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:46.382128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:46.382139 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:46.384148 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.384511 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.384513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.384514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.384615 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.384838 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.384839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.384839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.385072 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:46.385227 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:46.385344 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 22:44:46.385483 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260926 22:44:46.386703 18697 catalog_manager.cc:1347] Generated new cluster ID: 666c11b35bac4057aaf226e52551f623
I20260926 22:44:46.386752 18697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:46.397362 18697 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:46.398061 18697 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:46.411193 18697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17cf7abb44842928a3c958683bc3d58: Generated new TSK 0
I20260926 22:44:46.411381 18697 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:46.437460 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37743
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:46.519603 18706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:46.519800 18706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:46.519824 18706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:46.520895 18706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:46.520927 18706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:46.520952 18706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:46.522356 18706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:46.522532 18706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:46.522755 18706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:46.527520 18714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:46.527565 18711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:46.527520 18712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:46.527740 18706 server_base.cc:1034] running on GCE node
I20260926 22:44:46.527971 18706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:46.528204 18706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:46.529377 18706 hybrid_clock.cc:648] HybridClock initialized: now 1790462686529355 us; error 36 us; skew 500 ppm
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:46.531782 18706 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:46.532222 18706 webserver.cc:460] Webserver started at http://127.14.184.193:44691/ using document root <none> and password file <none>
I20260926 22:44:46.532437 18706 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:46.532500 18706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:46.532653 18706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:46.533610 18706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "9b6e7fa8c9cd41f6a64a88cd0508159b"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "78443bcbebf606dc50448f5bcfec14c4"
server_key_iv: "99eec1477ed45209c82aff7d5b6c94c3"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.533949 18706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "9b6e7fa8c9cd41f6a64a88cd0508159b"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "78443bcbebf606dc50448f5bcfec14c4"
server_key_iv: "99eec1477ed45209c82aff7d5b6c94c3"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.535295 18706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:46.535960 18721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.536123 18706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:46.536213 18706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "9b6e7fa8c9cd41f6a64a88cd0508159b"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "78443bcbebf606dc50448f5bcfec14c4"
server_key_iv: "99eec1477ed45209c82aff7d5b6c94c3"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.536286 18706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:46.555923 18706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:46.557595 18706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:46.557739 18706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:46.557950 18706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:46.558225 18706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:46.558275 18706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.558313 18706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:46.558338 18706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.564244 18706 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:35387
I20260926 22:44:46.564287 18834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:35387 every 8 connection(s)
I20260926 22:44:46.564594 18706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:46.572860 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.565248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:51559 (local address 127.14.184.254:37743)
0926 22:44:46.565300 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:46.565302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:46.565525 (+ 223us) server_negotiation.cc:407] Connection header received
0926 22:44:46.566392 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:46.566393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:46.566403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:46.566451 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:46.567045 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.567527 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.568445 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.568637 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.568687 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.570657 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:46.570658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:46.570659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:46.570664 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:46.571807 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.572166 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.572167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.572167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.572213 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.572404 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.572404 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.572405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.572493 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:46.572540 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:46.572736 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 22:44:46.572799 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:46.572994 18837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.565236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37743 (local address 127.14.184.193:51559)
0926 22:44:46.565401 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:46.565434 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:46.566294 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:46.566523 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:46.566527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:46.566769 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:46.566963 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:46.566967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.567598 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.567599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:46.568376 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:46.568378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.568703 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.568704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:46.568794 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.569280 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:46.569289 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:46.570574 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:46.571872 (+ 1298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:46.571878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:46.571879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:46.572108 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:46.572264 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:46.572264 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:46.572264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:46.572357 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:46.572541 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:46.572544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:46.572641 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 22:44:46.572715 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 22:44:46.572818 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260926 22:44:46.573434 18835 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37743
I20260926 22:44:46.573514 18835 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:46.573693 18835 heartbeater.cc:511] Master 127.14.184.254:37743 requested a full tablet report, sending...
I20260926 22:44:46.573691 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18706
I20260926 22:44:46.573787 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:46.573973 15075 external_mini_cluster.cc:1428] Setting key 526e11e1c1dc2cf67a6ea571e5c63eee
I20260926 22:44:46.574285 18643 ts_manager.cc:194] Registered new tserver with Master: 9b6e7fa8c9cd41f6a64a88cd0508159b (127.14.184.193:35387)
I20260926 22:44:46.575189 18643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:51559
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:46.595608 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37743
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:46.673887 18840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:46.674108 18840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:46.674142 18840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:46.675307 18840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:46.675380 18840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:46.675422 18840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:46.676911 18840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:46.677189 18840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:46.677412 18840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:46.682261 18845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:46.682343 18840 server_base.cc:1034] running on GCE node
W20260926 22:44:46.682272 18846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:46.682261 18848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:46.682765 18840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:46.683048 18840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:46.684201 18840 hybrid_clock.cc:648] HybridClock initialized: now 1790462686684189 us; error 26 us; skew 500 ppm
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:46.686978 18840 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:46.687536 18840 webserver.cc:460] Webserver started at http://127.14.184.194:42139/ using document root <none> and password file <none>
I20260926 22:44:46.687830 18840 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:46.687901 18840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:46.688051 18840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:46.689502 18840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "436052232aa8400baf70b4c1a4684cc3"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "77c4a7ffd53437c28b87d5266620a2ae"
server_key_iv: "fb0f884a338c2de44681340d5af035c1"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.689878 18840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "436052232aa8400baf70b4c1a4684cc3"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "77c4a7ffd53437c28b87d5266620a2ae"
server_key_iv: "fb0f884a338c2de44681340d5af035c1"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.691238 18840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260926 22:44:46.691926 18855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.692097 18840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.692178 18840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "436052232aa8400baf70b4c1a4684cc3"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "77c4a7ffd53437c28b87d5266620a2ae"
server_key_iv: "fb0f884a338c2de44681340d5af035c1"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.692250 18840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:46.711380 18840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:46.713160 18840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:46.713301 18840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:46.713505 18840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:46.713790 18840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:46.713837 18840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.713872 18840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:46.713898 18840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.719429 18840 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:40353
I20260926 22:44:46.719529 18968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:40353 every 8 connection(s)
I20260926 22:44:46.719790 18840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:46.720134 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18840
I20260926 22:44:46.720250 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:46.720463 15075 external_mini_cluster.cc:1428] Setting key 5dee8dd5ff1e1de8a1adff0c4c0a8884
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
I20260926 22:44:46.730747 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.720565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:33689 (local address 127.14.184.254:37743)
0926 22:44:46.720610 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:46.720611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:46.720896 (+ 285us) server_negotiation.cc:407] Connection header received
0926 22:44:46.722126 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:46.722127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:46.722137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:46.722177 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:46.723006 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.723795 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.725372 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.725559 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.725616 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.728002 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:46.728003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:46.728004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:46.728011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:46.729372 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.729765 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.729766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.729766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.729827 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.730095 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.730095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.730096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.730202 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:46.730272 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:46.730538 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 22:44:46.730618 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:46.730983 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.720545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37743 (local address 127.14.184.194:33689)
0926 22:44:46.720764 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:46.720803 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:46.722020 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:46.722245 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:46.722250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:46.722642 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:46.722919 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:46.722925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.723898 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.723900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:46.725283 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:46.725286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.725642 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.725643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:46.725801 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.726302 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:46.726313 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:46.727909 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:46.729450 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:46.729457 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:46.729459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:46.729676 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:46.729904 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:46.729905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:46.729905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:46.730024 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:46.730264 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:46.730270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:46.730409 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 22:44:46.730507 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 22:44:46.730727 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
I20260926 22:44:46.731446 18969 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37743
I20260926 22:44:46.731537 18969 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:46.731727 18969 heartbeater.cc:511] Master 127.14.184.254:37743 requested a full tablet report, sending...
I20260926 22:44:46.732137 18643 ts_manager.cc:194] Registered new tserver with Master: 436052232aa8400baf70b4c1a4684cc3 (127.14.184.194:40353)
I20260926 22:44:46.732663 18643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:33689
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:46.745275 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37743
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:46.823467 18974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:46.823714 18974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:46.823760 18974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:46.824875 18974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:46.824934 18974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:46.824978 18974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:46.826424 18974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:46.826704 18974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:46.826936 18974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:46.832175 18982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:46.832222 18980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:46.832202 18979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:46.832367 18974 server_base.cc:1034] running on GCE node
I20260926 22:44:46.832564 18974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:46.832824 18974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:46.833992 18974 hybrid_clock.cc:648] HybridClock initialized: now 1790462686833981 us; error 27 us; skew 500 ppm
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:46.836654 18974 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:46.837067 18974 webserver.cc:460] Webserver started at http://127.14.184.195:45933/ using document root <none> and password file <none>
I20260926 22:44:46.837255 18974 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:46.837296 18974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:46.837381 18974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:46.838418 18974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "d501a2919b554aed8fff4d32793a9f20"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "a181fa0df4f4c0dc4acc8e6b977a1447"
server_key_iv: "b6de981c9c912a4e2e386af81e98227e"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.838798 18974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "d501a2919b554aed8fff4d32793a9f20"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "a181fa0df4f4c0dc4acc8e6b977a1447"
server_key_iv: "b6de981c9c912a4e2e386af81e98227e"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.840525 18974 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260926 22:44:46.841310 18989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.841521 18974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:46.841615 18974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "d501a2919b554aed8fff4d32793a9f20"
format_stamp: "Formatted at 2026-09-26 22:44:46 on dist-test-slave-3986"
server_key: "a181fa0df4f4c0dc4acc8e6b977a1447"
server_key_iv: "b6de981c9c912a4e2e386af81e98227e"
server_key_version: "encryptionkey@0"
I20260926 22:44:46.841683 18974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:46.851693 18974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:46.854125 18974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:46.854285 18974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:46.854507 18974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:46.854890 18974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:46.854950 18974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.854991 18974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:46.855018 18974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:46.861756 18974 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:35905
I20260926 22:44:46.861789 19102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:35905 every 8 connection(s)
I20260926 22:44:46.862100 18974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:46.869508 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 18974
I20260926 22:44:46.869627 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:46.869830 15075 external_mini_cluster.cc:1428] Setting key 8babd027dedeeaf660e6a441bd503e6d
I20260926 22:44:46.871080 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.862809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:47331 (local address 127.14.184.254:37743)
0926 22:44:46.862851 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:46.862853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:46.863063 (+ 210us) server_negotiation.cc:407] Connection header received
0926 22:44:46.863909 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:46.863910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:46.863920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:46.863974 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:46.864512 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.865004 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.865967 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.866152 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.866209 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.868211 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:46.868212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:46.868212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:46.868219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:46.869410 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.869894 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.869895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.869896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.869978 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.870300 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.870301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.870301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.870438 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:46.870506 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:46.870932 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 22:44:46.871013 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:44:46.871235 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.862804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37743 (local address 127.14.184.195:47331)
0926 22:44:46.862940 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:46.862974 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:46.863820 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:46.864026 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:46.864030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:46.864256 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:46.864439 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:46.864443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.865091 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.865092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:46.865887 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:46.865889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.866230 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.866231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:46.866324 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.866828 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:46.866838 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:46.868131 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:46.869491 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:46.869498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:46.869500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:46.869814 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:46.870053 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:46.870053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:46.870054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:46.870225 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:46.870511 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:46.870518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:46.870675 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 22:44:46.870890 (+ 215us) client_negotiation.cc:240] Negotiation successful
0926 22:44:46.871046 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260926 22:44:46.871742 19103 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37743
I20260926 22:44:46.871858 19103 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:46.872025 19103 heartbeater.cc:511] Master 127.14.184.254:37743 requested a full tablet report, sending...
I20260926 22:44:46.872371 18643 ts_manager.cc:194] Registered new tserver with Master: d501a2919b554aed8fff4d32793a9f20 (127.14.184.195:35905)
I20260926 22:44:46.872727 18643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:47331
I20260926 22:44:46.881865 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:46.904994 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.897018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.14.184.193:35387)
0926 22:44:46.897179 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:46.897193 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:46.897202 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:46.897225 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:46.897227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:46.897240 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:46.897322 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:46.897924 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.898657 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.899595 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.899957 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.900065 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.902347 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:46.902355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:46.902356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:46.902364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:46.903783 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.904157 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.904160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.904161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.904227 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.904428 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.904429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.904430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.904565 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:46.904713 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:46.904796 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 22:44:46.904874 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:46.912760 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:46.906233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.14.184.254:37743)
0926 22:44:46.906348 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:46.906349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:46.906424 (+ 75us) server_negotiation.cc:407] Connection header received
0926 22:44:46.906464 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:46.906465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:46.906474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:46.906524 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:46.907100 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.907576 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.908229 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:46.908412 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:46.908461 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:46.910485 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:46.910486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:46.910486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:46.910493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:46.911824 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.912111 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.912111 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.912111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.912161 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:46.912317 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:46.912318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:46.912318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:46.912449 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:46.912522 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:46.912607 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 22:44:46.912663 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 22:44:46.913023 18643 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36884
I20260926 22:44:46.913393 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18706
I20260926 22:44:46.918305 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18840
I20260926 22:44:46.922384 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18974
I20260926 22:44:46.926709 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 18611
2026-09-26T22:44:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1807 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[19115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[19115](info): set up 2 sockets
Sep 26 22:44:46 dist-test-slave-3986 krb5kdc[19115](info): commencing operation
krb5kdc: starting...
W20260926 22:44:47.958961 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:48.069329 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37119
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:45887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:37119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:48.147900 19128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:48.148111 19128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:48.148134 19128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:48.149298 19128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:48.149339 19128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:48.149354 19128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:48.149405 19128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:48.149433 19128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:48.151328 19128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:37119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:48.151594 19128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:48.151825 19128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:48.156514 19133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.156558 19134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.156514 19136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:48.156767 19128 server_base.cc:1034] running on GCE node
I20260926 22:44:48.156978 19128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:48.157202 19128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:48.158365 19128 hybrid_clock.cc:648] HybridClock initialized: now 1790462688158360 us; error 35 us; skew 500 ppm
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:48.160774 19128 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:48.161094 19128 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:48.161140 19128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:48.161273 19128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:48.162274 19128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "e9b64d63bf6b48239e4216e656a0e394"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "8285dc9d9dbc6e99e5c13c7c37b70320"
server_key_iv: "d38b8eb1da57bc2eec651a5cdcac4992"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.162657 19128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "e9b64d63bf6b48239e4216e656a0e394"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "8285dc9d9dbc6e99e5c13c7c37b70320"
server_key_iv: "d38b8eb1da57bc2eec651a5cdcac4992"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.163988 19128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:48.164680 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.164858 19128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:48.164938 19128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "e9b64d63bf6b48239e4216e656a0e394"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "8285dc9d9dbc6e99e5c13c7c37b70320"
server_key_iv: "d38b8eb1da57bc2eec651a5cdcac4992"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.165027 19128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:48.173079 19128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:48.174889 19128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:48.175037 19128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:48.178606 19128 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:37119
I20260926 22:44:48.178633 19194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:37119 every 8 connection(s)
I20260926 22:44:48.178954 19128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:48.179865 19195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:48.182066 19195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394: Bootstrap starting.
I20260926 22:44:48.182639 19195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:48.182966 19195 log.cc:824] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:48.183579 19195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394: No bootstrap required, opened a new log
I20260926 22:44:48.183574 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19128
I20260926 22:44:48.183681 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:48.183833 15075 external_mini_cluster.cc:1428] Setting key a8aff6b7b79644b3cfeb16561d9d290a
I20260926 22:44:48.186337 19195 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b64d63bf6b48239e4216e656a0e394" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37119 } }
I20260926 22:44:48.186497 19195 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:48.186528 19195 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b64d63bf6b48239e4216e656a0e394, State: Initialized, Role: FOLLOWER
I20260926 22:44:48.186653 19195 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b64d63bf6b48239e4216e656a0e394" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37119 } }
I20260926 22:44:48.186750 19195 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:48.186779 19195 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:48.186820 19195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:48.187409 19195 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b64d63bf6b48239e4216e656a0e394" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37119 } }
I20260926 22:44:48.187520 19195 leader_election.cc:302] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9b64d63bf6b48239e4216e656a0e394; no voters:
I20260926 22:44:48.187697 19195 leader_election.cc:288] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:48.187726 19200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:48.187845 19200 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [term 1 LEADER]: Becoming Leader. State: Replica: e9b64d63bf6b48239e4216e656a0e394, State: Running, Role: LEADER
I20260926 22:44:48.187969 19195 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:48.187989 19200 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b64d63bf6b48239e4216e656a0e394" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37119 } }
I20260926 22:44:48.189639 19200 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9b64d63bf6b48239e4216e656a0e394. Latest consensus state: current_term: 1 leader_uuid: "e9b64d63bf6b48239e4216e656a0e394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b64d63bf6b48239e4216e656a0e394" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37119 } } }
I20260926 22:44:48.189785 19200 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:48.189857 19202 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9b64d63bf6b48239e4216e656a0e394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9b64d63bf6b48239e4216e656a0e394" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37119 } } }
I20260926 22:44:48.189950 19202 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:48.190246 19214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:48.190762 19214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:48.192358 19214 catalog_manager.cc:1347] Generated new cluster ID: 352e5d91db894ece96bb6669d931ca56
I20260926 22:44:48.192437 19214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:48.194980 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.184310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.14.184.254:37119)
0926 22:44:48.184502 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.184511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.184541 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:44:48.184958 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.184966 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.185021 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.185162 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:48.185730 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.187110 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.188087 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.188741 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.188858 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.191569 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:48.191573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:48.191573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:48.191584 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:48.193453 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.193806 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.193809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.193809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.193902 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.194104 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.194105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.194105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.194334 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:48.194496 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.194590 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.194766 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260926 22:44:48.199824 19214 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:48.200510 19214 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:48.202718 19214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394: Generated new TSK 0
I20260926 22:44:48.202870 19214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:48.227378 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37119
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:45887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:48.306578 19221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:48.306835 19221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:48.306860 19221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:48.308017 19221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:48.308056 19221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:48.308082 19221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:48.309582 19221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:48.309811 19221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:48.310050 19221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:48.314908 19226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.314921 19227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:48.314954 19221 server_base.cc:1034] running on GCE node
W20260926 22:44:48.314924 19229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:48.315269 19221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:48.315564 19221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:48.316725 19221 hybrid_clock.cc:648] HybridClock initialized: now 1790462688316712 us; error 42 us; skew 500 ppm
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:48.319684 19221 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:48.320015 19221 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:48.320086 19221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:48.320219 19221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:48.321226 19221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "e4a8c7b565bc496d92f1ea5682901633"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "eda635d97b42cadff60a67e571ac898f"
server_key_iv: "c1d5f6f36f4751db873652f04be13763"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.321589 19221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "e4a8c7b565bc496d92f1ea5682901633"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "eda635d97b42cadff60a67e571ac898f"
server_key_iv: "c1d5f6f36f4751db873652f04be13763"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.322860 19221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:44:48.323565 19235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.323734 19221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:48.323829 19221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "e4a8c7b565bc496d92f1ea5682901633"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "eda635d97b42cadff60a67e571ac898f"
server_key_iv: "c1d5f6f36f4751db873652f04be13763"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.323908 19221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:48.334472 19221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:48.336248 19221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:48.336404 19221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:48.336620 19221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:48.336912 19221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:48.336966 19221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.337004 19221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:48.337031 19221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.342571 19221 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:43277
I20260926 22:44:48.342636 19348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:43277 every 8 connection(s)
I20260926 22:44:48.342941 19221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:48.351586 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.343645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:60277 (local address 127.14.184.254:37119)
0926 22:44:48.343728 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.343730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.343919 (+ 189us) server_negotiation.cc:407] Connection header received
0926 22:44:48.344431 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.344432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.344442 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.344486 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:48.345355 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.345860 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.346860 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.347060 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.347113 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.348975 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:48.348976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:48.348976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:48.348982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:48.350355 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.350734 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.350734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.350735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.350784 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.350994 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.350995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.350995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.351092 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:48.351142 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.351357 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.351483 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 22:44:48.351595 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.343589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37119 (local address 127.14.184.193:60277)
0926 22:44:48.343794 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.343832 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.344321 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.344562 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.344565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.344801 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:48.345282 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.345287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.345941 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.345942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.346791 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.346794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.347129 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.347130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.347222 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.347627 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:48.347634 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:48.348887 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:48.350434 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:48.350440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:48.350441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:48.350634 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:48.350842 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:48.350843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:48.350843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:48.350938 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:48.351146 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:48.351150 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:48.351252 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.351329 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.351446 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260926 22:44:48.351584 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19221
I20260926 22:44:48.351711 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:48.351882 15075 external_mini_cluster.cc:1428] Setting key c78c1ff35168e0f5dc204dcf5b86a3a5
I20260926 22:44:48.352262 19349 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37119
I20260926 22:44:48.352372 19349 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:48.352543 19349 heartbeater.cc:511] Master 127.14.184.254:37119 requested a full tablet report, sending...
I20260926 22:44:48.353096 19159 ts_manager.cc:194] Registered new tserver with Master: e4a8c7b565bc496d92f1ea5682901633 (127.14.184.193:43277)
I20260926 22:44:48.353752 19159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:60277
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:48.375209 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37119
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:45887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:48.453293 19354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:48.453485 19354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:48.453509 19354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:48.454701 19354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:48.454742 19354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:48.454769 19354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:48.456252 19354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:48.456494 19354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:48.456713 19354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:48.461521 19359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.461527 19360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.461858 19362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:48.462095 19354 server_base.cc:1034] running on GCE node
I20260926 22:44:48.462337 19354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:48.462604 19354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:48.463757 19354 hybrid_clock.cc:648] HybridClock initialized: now 1790462688463727 us; error 39 us; skew 500 ppm
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:48.466702 19354 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:48.467113 19354 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:48.467182 19354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:48.467344 19354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:48.468536 19354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "73e19acb7ee241efae9b93c72882d5d9"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "45598eb2c1883a877ed8383f6ea60f40"
server_key_iv: "0927b7f8d9c4235525745232a298f8a8"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.468969 19354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "73e19acb7ee241efae9b93c72882d5d9"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "45598eb2c1883a877ed8383f6ea60f40"
server_key_iv: "0927b7f8d9c4235525745232a298f8a8"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.470259 19354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:48.470950 19368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.471127 19354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:48.471216 19354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "73e19acb7ee241efae9b93c72882d5d9"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "45598eb2c1883a877ed8383f6ea60f40"
server_key_iv: "0927b7f8d9c4235525745232a298f8a8"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.471287 19354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:48.482124 19354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:48.484453 19354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:48.484607 19354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:48.484823 19354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:48.485134 19354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:48.485188 19354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.485229 19354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:48.485257 19354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.491454 19354 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:46771
I20260926 22:44:48.491490 19481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:46771 every 8 connection(s)
I20260926 22:44:48.491750 19354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:48.499403 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19354
I20260926 22:44:48.499531 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:48.499740 15075 external_mini_cluster.cc:1428] Setting key 6f73a498eba210ad54f21215448c256a
I20260926 22:44:48.500308 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.492470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:60235 (local address 127.14.184.254:37119)
0926 22:44:48.492522 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.492524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.492734 (+ 210us) server_negotiation.cc:407] Connection header received
0926 22:44:48.493171 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.493172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.493182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.493240 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:48.494080 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.494559 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.495556 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.495732 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.495781 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.497729 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:48.497731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:48.497731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:48.497737 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:48.498988 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.499370 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.499372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.499372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.499434 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.499702 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.499703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.499703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.499838 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:48.499902 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.500178 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.500244 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:48.500546 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.492461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37119 (local address 127.14.184.194:60235)
0926 22:44:48.492609 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.492643 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.493092 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.493293 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.493296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.493535 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:48.494008 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.494012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.494634 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.494634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.495485 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.495488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.495798 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.495799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.495890 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.496361 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:48.496368 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:48.497648 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:48.499062 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:48.499068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:48.499069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:48.499268 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:48.499499 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:48.499500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:48.499501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:48.499637 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:48.499893 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:48.499899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:48.500029 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.500143 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.500290 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":88,"threads_started":1}
I20260926 22:44:48.501160 19482 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37119
I20260926 22:44:48.501240 19482 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:48.501384 19482 heartbeater.cc:511] Master 127.14.184.254:37119 requested a full tablet report, sending...
I20260926 22:44:48.501762 19159 ts_manager.cc:194] Registered new tserver with Master: 73e19acb7ee241efae9b93c72882d5d9 (127.14.184.194:46771)
I20260926 22:44:48.502283 19159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:60235
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:48.523439 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37119
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:45887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:48.602939 19487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:48.603127 19487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:48.603149 19487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:48.604236 19487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:48.604269 19487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:48.604293 19487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:48.605670 19487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:48.605839 19487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:48.606000 19487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:48.610608 19495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.610608 19493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.610620 19492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:48.610832 19487 server_base.cc:1034] running on GCE node
I20260926 22:44:48.611094 19487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:48.611347 19487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:48.612501 19487 hybrid_clock.cc:648] HybridClock initialized: now 1790462688612477 us; error 36 us; skew 500 ppm
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:48.614856 19487 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:48.615180 19487 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:48.615249 19487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:48.615386 19487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:48.616371 19487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "92eec155a6c141189dcdfc3d085e8864"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "ac06ecea4a8b788684c984ea3c16da1a"
server_key_iv: "c6fc10208c58a2b9969449dd28b31cb7"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.616737 19487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "92eec155a6c141189dcdfc3d085e8864"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "ac06ecea4a8b788684c984ea3c16da1a"
server_key_iv: "c6fc10208c58a2b9969449dd28b31cb7"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.618021 19487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:48.618726 19501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.618911 19487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:48.619005 19487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "92eec155a6c141189dcdfc3d085e8864"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "ac06ecea4a8b788684c984ea3c16da1a"
server_key_iv: "c6fc10208c58a2b9969449dd28b31cb7"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.619088 19487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:48.656234 19487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:48.658044 19487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:48.658209 19487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:48.658444 19487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:48.658815 19487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:48.658874 19487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.658911 19487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:48.658939 19487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.664772 19487 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33953
I20260926 22:44:48.664837 19614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33953 every 8 connection(s)
I20260926 22:44:48.665110 19487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:48.668246 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19487
I20260926 22:44:48.668367 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:48.668553 15075 external_mini_cluster.cc:1428] Setting key 862cc6c060a152acaee3aec0163cf030
Sep 26 22:44:48 dist-test-slave-3986 krb5kdc[19115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462688, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:48.674335 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.665786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:36823 (local address 127.14.184.254:37119)
0926 22:44:48.665870 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.665873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.666024 (+ 151us) server_negotiation.cc:407] Connection header received
0926 22:44:48.666476 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.666478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.666487 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.666539 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:48.667484 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.668175 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.669423 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.669629 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.669709 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.671779 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:48.671781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:48.671781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:48.671789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:48.672949 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.673360 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.673362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.673362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.673422 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.673661 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.673662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.673663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.673778 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:48.673869 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.674169 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.674247 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 22:44:48.674511 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.665742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37119 (local address 127.14.184.195:36823)
0926 22:44:48.665893 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.665938 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.666392 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.666592 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.666596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.666923 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:48.667409 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.667414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.668265 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.668265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.669330 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.669332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.669718 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.669719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.669835 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.670318 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:48.670331 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:48.671684 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:48.673046 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:48.673053 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:48.673055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:48.673264 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:48.673488 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:48.673489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:48.673489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:48.673593 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:48.673862 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:48.673868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:48.674033 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.674134 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.674279 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260926 22:44:48.675019 19615 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37119
I20260926 22:44:48.675132 19615 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:48.675282 19615 heartbeater.cc:511] Master 127.14.184.254:37119 requested a full tablet report, sending...
I20260926 22:44:48.675691 19159 ts_manager.cc:194] Registered new tserver with Master: 92eec155a6c141189dcdfc3d085e8864 (127.14.184.195:33953)
I20260926 22:44:48.676062 19159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:36823
I20260926 22:44:48.680186 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:44:48.686991 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.681501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.14.184.254:37119)
0926 22:44:48.681558 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.681560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.681679 (+ 119us) server_negotiation.cc:407] Connection header received
0926 22:44:48.681713 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.681714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.681722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.681762 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:48.682284 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.682801 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.683560 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.683746 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.683796 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.684913 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:48.684914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:48.684914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:48.684919 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:48.686026 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.686311 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.686311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.686312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.686357 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:48.686510 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:48.686510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:48.686510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:48.686632 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:48.686768 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.686845 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.686902 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:44:48.688629 19159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:48.690485 19159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:48.697774 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.692605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43277 (local address 127.0.0.1:44284)
0926 22:44:48.692951 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.692975 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.693152 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.693460 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.693464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.693466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:48.693530 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.693531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.694921 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.694921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.696756 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.696759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.697447 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.697448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.697538 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.697552 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.697623 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.697685 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":38,"thread_start_us":51,"threads_started":1}
I20260926 22:44:48.698010 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.692815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44284 (local address 127.14.184.193:43277)
0926 22:44:48.693194 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.693199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.693210 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:48.693270 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.693272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.693288 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.693362 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:48.693621 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.694812 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.696854 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.697344 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.697446 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.697507 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.697885 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.697936 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":78,"threads_started":1}
I20260926 22:44:48.698042 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.692877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.14.184.195:33953)
0926 22:44:48.693056 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.693061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.693591 (+ 530us) server_negotiation.cc:407] Connection header received
0926 22:44:48.693707 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.693709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.693724 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.693773 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:48.693977 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.695009 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.696875 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.697602 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.697672 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.697716 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.697920 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.697958 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260926 22:44:48.698254 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.692626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33953 (local address 127.0.0.1:46250)
0926 22:44:48.693530 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.693541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.693618 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.693868 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.693869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.693870 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:48.693904 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.693905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.695108 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.695109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.696755 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.696757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.697760 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.697761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.697820 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.697825 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.698147 (+ 322us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.698189 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"thread_start_us":55,"threads_started":1}
I20260926 22:44:48.698764 19283 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc4cd20a1c94a8f87e2b80c25128c18 (DEFAULT_TABLE table=test-table [id=d16736eea9f94da69e3d28c630718fa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:48.699007 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.692605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:46771 (local address 127.0.0.1:46016)
0926 22:44:48.692889 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.693061 (+ 172us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.693243 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.693518 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.693519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.693521 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:48.693575 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.693576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.694854 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.694855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.697859 (+ 3004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.697861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.698784 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.698785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.698849 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.698853 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.698909 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.698944 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":935,"thread_start_us":47,"threads_started":1}
I20260926 22:44:48.699036 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.693011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46016 (local address 127.14.184.194:46771)
0926 22:44:48.693159 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.693163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.693174 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:48.693332 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.693335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.693350 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.693431 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:48.693652 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.694733 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.697946 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.698709 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.698776 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.698836 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.698921 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.698959 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260926 22:44:48.698954 19549 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc4cd20a1c94a8f87e2b80c25128c18 (DEFAULT_TABLE table=test-table [id=d16736eea9f94da69e3d28c630718fa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:48.699122 19283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc4cd20a1c94a8f87e2b80c25128c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:48.699210 19549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc4cd20a1c94a8f87e2b80c25128c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:48.702713 19630 tablet_bootstrap.cc:492] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: Bootstrap starting.
I20260926 22:44:48.703215 19416 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc4cd20a1c94a8f87e2b80c25128c18 (DEFAULT_TABLE table=test-table [id=d16736eea9f94da69e3d28c630718fa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:48.703403 19630 tablet_bootstrap.cc:654] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:48.703550 19416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc4cd20a1c94a8f87e2b80c25128c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:48.703850 19630 log.cc:824] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:48.704094 19629 tablet_bootstrap.cc:492] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Bootstrap starting.
I20260926 22:44:48.704584 19630 tablet_bootstrap.cc:492] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: No bootstrap required, opened a new log
I20260926 22:44:48.704609 19629 tablet_bootstrap.cc:654] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:48.704681 19630 ts_tablet_manager.cc:1397] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:48.705013 19629 log.cc:824] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:48.705641 19629 tablet_bootstrap.cc:492] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: No bootstrap required, opened a new log
I20260926 22:44:48.705694 19629 ts_tablet_manager.cc:1397] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 22:44:48.706821 19633 tablet_bootstrap.cc:492] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Bootstrap starting.
I20260926 22:44:48.707011 19630 raft_consensus.cc:348] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.707180 19630 raft_consensus.cc:374] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:48.707221 19630 raft_consensus.cc:729] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a8c7b565bc496d92f1ea5682901633, State: Initialized, Role: FOLLOWER
I20260926 22:44:48.707345 19630 consensus_queue.cc:260] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.707408 19633 tablet_bootstrap.cc:654] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:48.707405 19629 raft_consensus.cc:348] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.707574 19629 raft_consensus.cc:374] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:48.707609 19630 ts_tablet_manager.cc:1428] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:48.707615 19629 raft_consensus.cc:729] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eec155a6c141189dcdfc3d085e8864, State: Initialized, Role: FOLLOWER
I20260926 22:44:48.707729 19349 heartbeater.cc:503] Master 127.14.184.254:37119 was elected leader, sending a full tablet report...
I20260926 22:44:48.707777 19629 consensus_queue.cc:260] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.707970 19633 log.cc:824] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:48.708113 19629 ts_tablet_manager.cc:1428] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:44:48.708153 19615 heartbeater.cc:503] Master 127.14.184.254:37119 was elected leader, sending a full tablet report...
I20260926 22:44:48.708876 19633 tablet_bootstrap.cc:492] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: No bootstrap required, opened a new log
I20260926 22:44:48.708974 19633 ts_tablet_manager.cc:1397] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:48.710224 19633 raft_consensus.cc:348] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.710378 19633 raft_consensus.cc:374] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:48.710434 19633 raft_consensus.cc:729] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e19acb7ee241efae9b93c72882d5d9, State: Initialized, Role: FOLLOWER
I20260926 22:44:48.710536 19633 consensus_queue.cc:260] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.710826 19633 ts_tablet_manager.cc:1428] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:44:48.710826 19482 heartbeater.cc:503] Master 127.14.184.254:37119 was elected leader, sending a full tablet report...
W20260926 22:44:48.742280 19483 tablet.cc:2369] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:48.771857 19634 raft_consensus.cc:482] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:48.771976 19634 raft_consensus.cc:504] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.772399 19634 leader_election.cc:288] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73e19acb7ee241efae9b93c72882d5d9 (127.14.184.194:46771), 92eec155a6c141189dcdfc3d085e8864 (127.14.184.195:33953)
I20260926 22:44:48.775480 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.772657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:44287 (local address 127.14.184.194:46771)
0926 22:44:48.772701 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.772704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.772713 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:48.772774 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.772775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.772784 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.772871 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:48.773029 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.773563 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.774641 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.775114 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.775193 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.775204 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.775379 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.775413 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:48.775513 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.772559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:46771 (local address 127.14.184.193:44287)
0926 22:44:48.772631 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.772643 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.772701 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.772878 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.772879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.772880 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:48.772951 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.772955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.773663 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.773664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.774556 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.774558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.775200 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.775201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.775288 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.775291 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.775367 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.775406 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":3}
I20260926 22:44:48.775527 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.772676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59595 (local address 127.14.184.195:33953)
0926 22:44:48.772730 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.772732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.772896 (+ 164us) server_negotiation.cc:407] Connection header received
0926 22:44:48.772974 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.772975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.772984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.773055 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:48.773216 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.773689 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.774653 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.775162 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.775245 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.775253 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.775424 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.775460 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 22:44:48.775619 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.772565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33953 (local address 127.14.184.193:59595)
0926 22:44:48.772813 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.772822 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:48.772909 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:48.773057 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:48.773058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:48.773059 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:48.773155 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.773157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.773806 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.773807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.774557 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:48.774559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.775256 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.775257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:48.775332 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.775335 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:48.775411 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:44:48.775455 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":73,"thread_start_us":119,"threads_started":1}
I20260926 22:44:48.775904 19569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc4cd20a1c94a8f87e2b80c25128c18" candidate_uuid: "e4a8c7b565bc496d92f1ea5682901633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eec155a6c141189dcdfc3d085e8864" is_pre_election: true
I20260926 22:44:48.775913 19436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc4cd20a1c94a8f87e2b80c25128c18" candidate_uuid: "e4a8c7b565bc496d92f1ea5682901633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73e19acb7ee241efae9b93c72882d5d9" is_pre_election: true
I20260926 22:44:48.776049 19436 raft_consensus.cc:2454] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4a8c7b565bc496d92f1ea5682901633 in term 0.
I20260926 22:44:48.776052 19569 raft_consensus.cc:2454] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4a8c7b565bc496d92f1ea5682901633 in term 0.
I20260926 22:44:48.776262 19239 leader_election.cc:302] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73e19acb7ee241efae9b93c72882d5d9, e4a8c7b565bc496d92f1ea5682901633; no voters:
I20260926 22:44:48.776388 19634 raft_consensus.cc:2781] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:48.776443 19634 raft_consensus.cc:482] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:48.776466 19634 raft_consensus.cc:3037] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:48.777120 19634 raft_consensus.cc:504] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.777262 19634 leader_election.cc:288] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [CANDIDATE]: Term 1 election: Requested vote from peers 73e19acb7ee241efae9b93c72882d5d9 (127.14.184.194:46771), 92eec155a6c141189dcdfc3d085e8864 (127.14.184.195:33953)
I20260926 22:44:48.777446 19436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc4cd20a1c94a8f87e2b80c25128c18" candidate_uuid: "e4a8c7b565bc496d92f1ea5682901633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73e19acb7ee241efae9b93c72882d5d9"
I20260926 22:44:48.777534 19436 raft_consensus.cc:3037] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:48.777521 19569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc4cd20a1c94a8f87e2b80c25128c18" candidate_uuid: "e4a8c7b565bc496d92f1ea5682901633" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eec155a6c141189dcdfc3d085e8864"
I20260926 22:44:48.777606 19569 raft_consensus.cc:3037] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:48.778276 19436 raft_consensus.cc:2454] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4a8c7b565bc496d92f1ea5682901633 in term 1.
I20260926 22:44:48.778277 19569 raft_consensus.cc:2454] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4a8c7b565bc496d92f1ea5682901633 in term 1.
I20260926 22:44:48.778460 19238 leader_election.cc:302] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eec155a6c141189dcdfc3d085e8864, e4a8c7b565bc496d92f1ea5682901633; no voters:
I20260926 22:44:48.778566 19634 raft_consensus.cc:2781] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:48.778761 19634 raft_consensus.cc:686] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 1 LEADER]: Becoming Leader. State: Replica: e4a8c7b565bc496d92f1ea5682901633, State: Running, Role: LEADER
I20260926 22:44:48.778872 19634 consensus_queue.cc:237] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } }
I20260926 22:44:48.779641 19159 catalog_manager.cc:5495] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4a8c7b565bc496d92f1ea5682901633 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "e4a8c7b565bc496d92f1ea5682901633" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4a8c7b565bc496d92f1ea5682901633" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:48.789958 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.787678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.14.184.193:43277)
0926 22:44:48.787716 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.787718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.787748 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:44:48.787792 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.787792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.787807 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.787863 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:48.788059 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.788534 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.789293 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:48.789504 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:48.789556 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:48.789691 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:48.789783 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:48.789857 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.789878 (+ 21us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.789894 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:48.791675 19436 raft_consensus.cc:1261] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 1 FOLLOWER]: Refusing update from remote peer e4a8c7b565bc496d92f1ea5682901633: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:48.791743 19569 raft_consensus.cc:1261] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 1 FOLLOWER]: Refusing update from remote peer e4a8c7b565bc496d92f1ea5682901633: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:48.791990 19634 consensus_queue.cc:1035] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73e19acb7ee241efae9b93c72882d5d9" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 46771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:48.792109 19639 consensus_queue.cc:1035] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92eec155a6c141189dcdfc3d085e8864" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:48.796588 19158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:48.796684 19158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:48.797431 19158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9b64d63bf6b48239e4216e656a0e394: Sending DeleteTablet for 3 replicas of tablet 9bc4cd20a1c94a8f87e2b80c25128c18
I20260926 22:44:48.797753 19549 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc4cd20a1c94a8f87e2b80c25128c18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:48 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:46250
I20260926 22:44:48.797767 19416 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc4cd20a1c94a8f87e2b80c25128c18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:48 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:46016
I20260926 22:44:48.797792 19283 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc4cd20a1c94a8f87e2b80c25128c18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:48 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:44284
I20260926 22:44:48.797981 19653 tablet_replica.cc:330] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: stopping tablet replica
I20260926 22:44:48.798026 19651 tablet_replica.cc:330] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633: stopping tablet replica
I20260926 22:44:48.798075 19653 raft_consensus.cc:2229] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:48.798095 19651 raft_consensus.cc:2229] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:48.798163 19653 raft_consensus.cc:2258] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:48.798168 19651 raft_consensus.cc:2258] T 9bc4cd20a1c94a8f87e2b80c25128c18 P e4a8c7b565bc496d92f1ea5682901633 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:48.798254 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19221
I20260926 22:44:48.799759 19653 ts_tablet_manager.cc:1905] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:48.799965 19652 tablet_replica.cc:330] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: stopping tablet replica
I20260926 22:44:48.800066 19652 raft_consensus.cc:2229] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:48.800163 19652 raft_consensus.cc:2258] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:48.800500 19652 ts_tablet_manager.cc:1905] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:48.801775 19652 ts_tablet_manager.cc:1918] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:48.801839 19653 ts_tablet_manager.cc:1918] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:48.801872 19652 log.cc:1196] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/9bc4cd20a1c94a8f87e2b80c25128c18
I20260926 22:44:48.801923 19653 log.cc:1196] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/9bc4cd20a1c94a8f87e2b80c25128c18
I20260926 22:44:48.802146 19652 ts_tablet_manager.cc:1939] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 92eec155a6c141189dcdfc3d085e8864: Deleting consensus metadata
I20260926 22:44:48.802198 19653 ts_tablet_manager.cc:1939] T 9bc4cd20a1c94a8f87e2b80c25128c18 P 73e19acb7ee241efae9b93c72882d5d9: Deleting consensus metadata
I20260926 22:44:48.802579 19146 catalog_manager.cc:4841] TS 73e19acb7ee241efae9b93c72882d5d9 (127.14.184.194:46771): tablet 9bc4cd20a1c94a8f87e2b80c25128c18 (table test-table [id=d16736eea9f94da69e3d28c630718fa6]) successfully deleted
I20260926 22:44:48.802559 19145 catalog_manager.cc:4841] TS 92eec155a6c141189dcdfc3d085e8864 (127.14.184.195:33953): tablet 9bc4cd20a1c94a8f87e2b80c25128c18 (table test-table [id=d16736eea9f94da69e3d28c630718fa6]) successfully deleted
W20260926 22:44:48.804574 19144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:43277 (error 108)
I20260926 22:44:48.805091 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19354
I20260926 22:44:48.805104 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.804969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43277 (local address 127.0.0.1:44310)
0926 22:44:48.805032 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:48.805075 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:43277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 22:44:48.807298 19144 catalog_manager.cc:4584] TS e4a8c7b565bc496d92f1ea5682901633 (127.14.184.193:43277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9bc4cd20a1c94a8f87e2b80c25128c18: Network error: Client connection negotiation failed: client connection to 127.14.184.193:43277: connect: Connection refused (error 111)
I20260926 22:44:48.809746 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19487
I20260926 22:44:48.814056 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19128
2026-09-26T22:44:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1887 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T22:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:48Z Disabled control of system clock
I20260926 22:44:48.840884 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:40993
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:37511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:40993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 22:44:48.919119 19658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:48.919310 19658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:48.919332 19658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:48.920360 19658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:48.920401 19658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:48.920414 19658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:48.920426 19658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:48.920439 19658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:48.922217 19658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:37511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:40993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:48.922407 19658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:48.922626 19658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:48.927289 19664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.927361 19666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:48.927328 19663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:48.927491 19658 server_base.cc:1034] running on GCE node
I20260926 22:44:48.927716 19658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:48.928023 19658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:48.929178 19658 hybrid_clock.cc:648] HybridClock initialized: now 1790462688929160 us; error 37 us; skew 500 ppm
I20260926 22:44:48.930318 19658 webserver.cc:460] Webserver started at http://127.14.184.254:41631/ using document root <none> and password file <none>
I20260926 22:44:48.930557 19658 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:48.930621 19658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:48.930779 19658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:48.931778 19658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "ba77be74a0f349088ef81223d50e20fa"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "953723d866a1af4b12954f849c465533"
server_key_iv: "3d0a082b642a4ae4cbe27344eb50832c"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.932135 19658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "ba77be74a0f349088ef81223d50e20fa"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "953723d866a1af4b12954f849c465533"
server_key_iv: "3d0a082b642a4ae4cbe27344eb50832c"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.933413 19658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:48.934167 19672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:48.934335 19658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:48.934438 19658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "ba77be74a0f349088ef81223d50e20fa"
format_stamp: "Formatted at 2026-09-26 22:44:48 on dist-test-slave-3986"
server_key: "953723d866a1af4b12954f849c465533"
server_key_iv: "3d0a082b642a4ae4cbe27344eb50832c"
server_key_version: "encryptionkey@0"
I20260926 22:44:48.934520 19658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:48.937657 19658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:48.937908 19658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:48.938046 19658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:48.941741 19658 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:40993
I20260926 22:44:48.941771 19724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:40993 every 8 connection(s)
I20260926 22:44:48.942080 19658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:48.942708 19725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:48.944619 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19658
I20260926 22:44:48.944720 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:48.944909 15075 external_mini_cluster.cc:1428] Setting key bf1d09f24c8b856138bf65aeb66c7f19
I20260926 22:44:48.945767 19725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa: Bootstrap starting.
I20260926 22:44:48.946446 19725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:48.946808 19725 log.cc:824] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:48.947590 19725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa: No bootstrap required, opened a new log
I20260926 22:44:48.947638 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:48.945446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.14.184.254:40993)
0926 22:44:48.945627 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:48.945634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:48.945674 (+ 40us) server_negotiation.cc:407] Connection header received
0926 22:44:48.946145 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:48.946152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:48.946224 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:48.946380 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:48.947162 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:48.947166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:48.947166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:48.947171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:48.947219 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:48.947233 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:48.947308 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:48.947382 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:44:48.947449 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260926 22:44:48.949035 19725 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba77be74a0f349088ef81223d50e20fa" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40993 } }
I20260926 22:44:48.949180 19725 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:48.949226 19725 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba77be74a0f349088ef81223d50e20fa, State: Initialized, Role: FOLLOWER
I20260926 22:44:48.949345 19725 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba77be74a0f349088ef81223d50e20fa" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40993 } }
I20260926 22:44:48.949682 19725 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:48.949730 19725 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:48.949800 19725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:48.950278 19725 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba77be74a0f349088ef81223d50e20fa" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40993 } }
I20260926 22:44:48.950397 19725 leader_election.cc:302] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba77be74a0f349088ef81223d50e20fa; no voters:
I20260926 22:44:48.950563 19725 leader_election.cc:288] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:48.950592 19730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:48.950778 19730 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [term 1 LEADER]: Becoming Leader. State: Replica: ba77be74a0f349088ef81223d50e20fa, State: Running, Role: LEADER
I20260926 22:44:48.950843 19725 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:48.950875 19730 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba77be74a0f349088ef81223d50e20fa" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40993 } }
I20260926 22:44:48.951203 19731 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba77be74a0f349088ef81223d50e20fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba77be74a0f349088ef81223d50e20fa" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40993 } } }
I20260926 22:44:48.951217 19732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba77be74a0f349088ef81223d50e20fa. Latest consensus state: current_term: 1 leader_uuid: "ba77be74a0f349088ef81223d50e20fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba77be74a0f349088ef81223d50e20fa" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 40993 } } }
I20260926 22:44:48.951367 19732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:48.951579 19731 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:48.951747 19737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:48.952235 19737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:48.953673 19737 catalog_manager.cc:1347] Generated new cluster ID: 2d5579a3e0fc458b9f60138bef32dae0
I20260926 22:44:48.953747 19737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:48.960414 19737 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:48.960556 19737 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:48.967305 19737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa: Generated new TSK 0
I20260926 22:44:48.967482 19737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:44:48.973224 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40993
--builtin_ntp_servers=127.14.184.212:37511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 22:44:49.063017 19749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:49.063232 19749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:49.063280 19749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:49.064443 19749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:49.064507 19749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:49.064549 19749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:49.065876 19749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:37511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:49.066126 19749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:49.066418 19749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:49.071933 19757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:49.072009 19754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:49.072013 19755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.072381 19749 server_base.cc:1034] running on GCE node
I20260926 22:44:49.072571 19749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:49.072839 19749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:49.074013 19749 hybrid_clock.cc:648] HybridClock initialized: now 1790462689073994 us; error 35 us; skew 500 ppm
I20260926 22:44:49.075304 19749 webserver.cc:460] Webserver started at http://127.14.184.193:46803/ using document root <none> and password file <none>
I20260926 22:44:49.075558 19749 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:49.075627 19749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:49.075778 19749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:49.076798 19749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "febf63e62333478bbabb13c37fdb00af"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "01bfdb974282cd1c4ff9f0f2c50af0a0"
server_key_iv: "5a2a715d3c9ace682e7e4c4d18e31017"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.077163 19749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "febf63e62333478bbabb13c37fdb00af"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "01bfdb974282cd1c4ff9f0f2c50af0a0"
server_key_iv: "5a2a715d3c9ace682e7e4c4d18e31017"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.078491 19749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:44:49.079324 19763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.079545 19749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:49.079653 19749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "febf63e62333478bbabb13c37fdb00af"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "01bfdb974282cd1c4ff9f0f2c50af0a0"
server_key_iv: "5a2a715d3c9ace682e7e4c4d18e31017"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.079743 19749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:49.083119 19749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:49.083379 19749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:49.083523 19749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:49.083730 19749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:49.084019 19749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:49.084074 19749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.084112 19749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:49.084138 19749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.089589 19749 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:44047
I20260926 22:44:49.089654 19876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:44047 every 8 connection(s)
I20260926 22:44:49.089972 19749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:49.091516 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19749
I20260926 22:44:49.091626 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:49.091833 15075 external_mini_cluster.cc:1428] Setting key 2b95f1bd68a8e73665d3dad8ef20da8a
I20260926 22:44:49.092748 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40993
--builtin_ntp_servers=127.14.184.212:37511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 22:44:49.093142 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.090708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:54393 (local address 127.14.184.254:40993)
0926 22:44:49.090757 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.090760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.091002 (+ 242us) server_negotiation.cc:407] Connection header received
0926 22:44:49.091437 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.091438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.091444 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.091529 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.092824 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.092826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.092826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.092831 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.092845 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.092849 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.092904 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.093027 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.093043 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:44:49.093297 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.090676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40993 (local address 127.14.184.193:54393)
0926 22:44:49.090887 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.090919 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.091336 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.091601 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.091605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.091616 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.092675 (+ 1059us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.092676 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.092710 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.092736 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.092910 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.092915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.092918 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.093002 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.093065 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260926 22:44:49.093760 19877 heartbeater.cc:347] Connected to a master server at 127.14.184.254:40993
I20260926 22:44:49.093859 19877 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:49.094002 19877 heartbeater.cc:511] Master 127.14.184.254:40993 requested a full tablet report, sending...
I20260926 22:44:49.094522 19689 ts_manager.cc:194] Registered new tserver with Master: febf63e62333478bbabb13c37fdb00af (127.14.184.193:44047)
W20260926 22:44:49.171579 19880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:49.171764 19880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:49.171785 19880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:49.172850 19880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:49.172883 19880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:49.172909 19880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:49.174126 19880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:37511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:49.174294 19880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:49.174516 19880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:49.179404 19888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:49.179430 19886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.179503 19880 server_base.cc:1034] running on GCE node
W20260926 22:44:49.179407 19885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.179802 19880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:49.180063 19880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:49.181223 19880 hybrid_clock.cc:648] HybridClock initialized: now 1790462689181212 us; error 38 us; skew 500 ppm
I20260926 22:44:49.182492 19880 webserver.cc:460] Webserver started at http://127.14.184.194:43665/ using document root <none> and password file <none>
I20260926 22:44:49.182783 19880 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:49.182852 19880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:49.182988 19880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:49.184075 19880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "11a4c19f626b49b897f1ddd07b91bf4c"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "c75eed8d912dbb5de3c754a200393450"
server_key_iv: "841d65fedffe96baf9216858cdb32c6a"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.184435 19880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "11a4c19f626b49b897f1ddd07b91bf4c"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "c75eed8d912dbb5de3c754a200393450"
server_key_iv: "841d65fedffe96baf9216858cdb32c6a"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.185720 19880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:44:49.186468 19894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.186640 19880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:49.186753 19880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "11a4c19f626b49b897f1ddd07b91bf4c"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "c75eed8d912dbb5de3c754a200393450"
server_key_iv: "841d65fedffe96baf9216858cdb32c6a"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.186842 19880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:49.190007 19880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:49.190279 19880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:49.190428 19880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:49.190639 19880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:49.190953 19880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:49.191007 19880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.191045 19880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:49.191072 19880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.196422 19880 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:41833
I20260926 22:44:49.196491 20007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:41833 every 8 connection(s)
I20260926 22:44:49.196790 19880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:49.197381 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 19880
I20260926 22:44:49.197467 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:49.197672 15075 external_mini_cluster.cc:1428] Setting key ed74c7a7bb079177c9ed7e882a131e7a
I20260926 22:44:49.198800 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40993
--builtin_ntp_servers=127.14.184.212:37511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 22:44:49.200063 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.197476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:33685 (local address 127.14.184.254:40993)
0926 22:44:49.197547 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.197549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.197790 (+ 241us) server_negotiation.cc:407] Connection header received
0926 22:44:49.198386 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.198387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.198393 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.198433 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.199829 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.199830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.199830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.199834 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.199849 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.199853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.199909 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.199998 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.200011 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:44:49.200294 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.197447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40993 (local address 127.14.184.194:33685)
0926 22:44:49.197657 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.197699 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.198271 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.198515 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.198520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.198530 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.199666 (+ 1136us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.199667 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.199704 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.199737 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.199922 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.199926 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.199929 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.200003 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.200065 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260926 22:44:49.200778 20008 heartbeater.cc:347] Connected to a master server at 127.14.184.254:40993
I20260926 22:44:49.200872 20008 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:49.201018 20008 heartbeater.cc:511] Master 127.14.184.254:40993 requested a full tablet report, sending...
I20260926 22:44:49.201349 19689 ts_manager.cc:194] Registered new tserver with Master: 11a4c19f626b49b897f1ddd07b91bf4c (127.14.184.194:41833)
W20260926 22:44:49.279074 20011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:49.279273 20011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:49.279294 20011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:49.280388 20011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:49.280422 20011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:49.280447 20011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:49.281708 20011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:37511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:49.281891 20011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:49.282124 20011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:49.287052 20019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.287189 20011 server_base.cc:1034] running on GCE node
W20260926 22:44:49.287124 20017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:49.287110 20016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.287542 20011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:49.287750 20011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:49.288956 20011 hybrid_clock.cc:648] HybridClock initialized: now 1790462689288941 us; error 25 us; skew 500 ppm
I20260926 22:44:49.290254 20011 webserver.cc:460] Webserver started at http://127.14.184.195:39045/ using document root <none> and password file <none>
I20260926 22:44:49.290541 20011 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:49.290609 20011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:49.290786 20011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:49.291782 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "56100b08e5f34f23a0cd17d1f3eb5536"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "33d861f9cc93c643b856d589db38966c"
server_key_iv: "71377b62ef816c24acf3de8d8ce2df72"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.292126 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "56100b08e5f34f23a0cd17d1f3eb5536"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "33d861f9cc93c643b856d589db38966c"
server_key_iv: "71377b62ef816c24acf3de8d8ce2df72"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.293373 20011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:49.294114 20025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.294281 20011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:49.294381 20011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "56100b08e5f34f23a0cd17d1f3eb5536"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "33d861f9cc93c643b856d589db38966c"
server_key_iv: "71377b62ef816c24acf3de8d8ce2df72"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.294459 20011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:49.297644 20011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:49.297907 20011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:49.298044 20011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:49.298257 20011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:49.298547 20011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:49.298600 20011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.298637 20011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:49.298681 20011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.304342 20011 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:41693
I20260926 22:44:49.304406 20138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:41693 every 8 connection(s)
I20260926 22:44:49.304774 20011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:49.307476 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.305459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:50341 (local address 127.14.184.254:40993)
0926 22:44:49.305546 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.305549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.305734 (+ 185us) server_negotiation.cc:407] Connection header received
0926 22:44:49.306150 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.306152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.306156 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.306207 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.307298 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.307299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.307299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.307303 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.307316 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.307319 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.307367 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.307414 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.307425 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 22:44:49.307660 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.305421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:40993 (local address 127.14.184.195:50341)
0926 22:44:49.305618 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.305648 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.306071 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.306256 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.306259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.306266 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.307162 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.307163 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.307190 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.307215 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.307365 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.307368 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.307370 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.307421 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.307465 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260926 22:44:49.308125 20139 heartbeater.cc:347] Connected to a master server at 127.14.184.254:40993
I20260926 22:44:49.308199 20139 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:49.308343 20139 heartbeater.cc:511] Master 127.14.184.254:40993 requested a full tablet report, sending...
I20260926 22:44:49.308707 19689 ts_manager.cc:194] Registered new tserver with Master: 56100b08e5f34f23a0cd17d1f3eb5536 (127.14.184.195:41693)
I20260926 22:44:49.314021 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20011
I20260926 22:44:49.314124 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:49.314307 15075 external_mini_cluster.cc:1428] Setting key 19f24bd3e6b9ec69927cffa3f112bc46
I20260926 22:44:49.315236 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:44:49.317454 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.316523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.14.184.254:40993)
0926 22:44:49.316583 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.316584 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.316668 (+ 84us) server_negotiation.cc:407] Connection header received
0926 22:44:49.316704 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.316705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.316708 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.316760 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.317230 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.317231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.317231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.317235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.317245 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.317249 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.317295 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.317334 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.317345 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 22:44:49.319073 19689 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:49.320538 19689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:49.324111 20149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.322552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:41833 (local address 127.0.0.1:36396)
0926 22:44:49.322966 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.322975 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.323020 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.323154 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.323155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.323157 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.323688 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.323688 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.323693 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.323702 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.323930 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.323936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.323939 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.324003 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.324017 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":70,"threads_started":1}
I20260926 22:44:49.324144 20150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.322780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.14.184.194:41833)
0926 22:44:49.322963 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.322968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.323018 (+ 50us) server_negotiation.cc:407] Connection header received
0926 22:44:49.323072 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.323075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.323083 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.323149 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.323791 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.323809 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.323809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.323813 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.323841 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.323858 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.323924 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.324033 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.324052 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260926 22:44:49.324363 20152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.322780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.14.184.195:41693)
0926 22:44:49.323003 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.323006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.323016 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:49.323039 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.323041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.323050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.323136 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.323941 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.323944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.323945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.323948 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.323998 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.324013 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.324090 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.324280 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.324294 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":114,"threads_started":1}
I20260926 22:44:49.324218 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.322529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:41693 (local address 127.0.0.1:47098)
0926 22:44:49.322629 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.322683 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.322733 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.323134 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.323140 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.323147 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.323633 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.323633 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.323658 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.323681 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.324089 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.324090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.324091 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.324138 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.324149 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":61,"threads_started":1}
I20260926 22:44:49.325171 20151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.322777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34118 (local address 127.14.184.193:44047)
0926 22:44:49.324173 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.324178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.324188 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:49.324213 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.324215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.324224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.324293 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.324927 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.324932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.324932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.324936 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.324963 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.324978 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.325029 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.325103 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.325115 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1358,"thread_start_us":114,"threads_started":1}
I20260926 22:44:49.325177 20148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.322543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44047 (local address 127.0.0.1:34118)
0926 22:44:49.322953 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.322965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.323014 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.324296 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.324297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.324300 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.324834 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.324835 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.324841 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.324854 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.325045 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.325045 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.325046 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.325102 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.325114 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":78,"threads_started":1}
I20260926 22:44:49.325213 19942 tablet_service.cc:1467] Processing CreateTablet for tablet ecc1bae05cbf4d1486d79d48bd7c47e9 (DEFAULT_TABLE table=test-table [id=88dd45d7553b44ab9880c9f53c7ff259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:49.325618 19942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc1bae05cbf4d1486d79d48bd7c47e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:49.325726 19811 tablet_service.cc:1467] Processing CreateTablet for tablet ecc1bae05cbf4d1486d79d48bd7c47e9 (DEFAULT_TABLE table=test-table [id=88dd45d7553b44ab9880c9f53c7ff259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:49.326037 19811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc1bae05cbf4d1486d79d48bd7c47e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:49.328321 20153 tablet_bootstrap.cc:492] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Bootstrap starting.
I20260926 22:44:49.328708 20073 tablet_service.cc:1467] Processing CreateTablet for tablet ecc1bae05cbf4d1486d79d48bd7c47e9 (DEFAULT_TABLE table=test-table [id=88dd45d7553b44ab9880c9f53c7ff259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:49.328853 20154 tablet_bootstrap.cc:492] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: Bootstrap starting.
I20260926 22:44:49.328984 20153 tablet_bootstrap.cc:654] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:49.329031 20073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc1bae05cbf4d1486d79d48bd7c47e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:49.329298 20153 log.cc:824] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:49.329596 20154 tablet_bootstrap.cc:654] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:49.329913 20154 log.cc:824] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:49.330153 20153 tablet_bootstrap.cc:492] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: No bootstrap required, opened a new log
I20260926 22:44:49.330228 20153 ts_tablet_manager.cc:1397] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:49.330549 20154 tablet_bootstrap.cc:492] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: No bootstrap required, opened a new log
I20260926 22:44:49.330627 20154 ts_tablet_manager.cc:1397] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:49.332204 20157 tablet_bootstrap.cc:492] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Bootstrap starting.
I20260926 22:44:49.332897 20157 tablet_bootstrap.cc:654] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:49.332774 20154 raft_consensus.cc:348] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.332952 20154 raft_consensus.cc:374] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:49.332989 20154 raft_consensus.cc:729] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febf63e62333478bbabb13c37fdb00af, State: Initialized, Role: FOLLOWER
I20260926 22:44:49.332882 20153 raft_consensus.cc:348] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.333045 20153 raft_consensus.cc:374] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:49.333079 20153 raft_consensus.cc:729] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a4c19f626b49b897f1ddd07b91bf4c, State: Initialized, Role: FOLLOWER
I20260926 22:44:49.333148 20154 consensus_queue.cc:260] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.333202 20153 consensus_queue.cc:260] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.333244 20157 log.cc:824] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:49.333452 20153 ts_tablet_manager.cc:1428] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:49.333470 20154 ts_tablet_manager.cc:1428] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:49.333592 20008 heartbeater.cc:503] Master 127.14.184.254:40993 was elected leader, sending a full tablet report...
I20260926 22:44:49.333886 19877 heartbeater.cc:503] Master 127.14.184.254:40993 was elected leader, sending a full tablet report...
I20260926 22:44:49.334012 20157 tablet_bootstrap.cc:492] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: No bootstrap required, opened a new log
I20260926 22:44:49.334069 20157 ts_tablet_manager.cc:1397] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:49.336159 20157 raft_consensus.cc:348] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.336280 20157 raft_consensus.cc:374] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:49.336318 20157 raft_consensus.cc:729] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56100b08e5f34f23a0cd17d1f3eb5536, State: Initialized, Role: FOLLOWER
I20260926 22:44:49.336440 20157 consensus_queue.cc:260] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.336717 20157 ts_tablet_manager.cc:1428] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:49.336725 20139 heartbeater.cc:503] Master 127.14.184.254:40993 was elected leader, sending a full tablet report...
W20260926 22:44:49.340452 19878 tablet.cc:2369] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:44:49.447325 20009 tablet.cc:2369] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:44:49.555289 20140 tablet.cc:2369] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:49.657074 20159 raft_consensus.cc:482] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:49.657258 20159 raft_consensus.cc:504] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.657668 20159 leader_election.cc:288] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56100b08e5f34f23a0cd17d1f3eb5536 (127.14.184.195:41693), 11a4c19f626b49b897f1ddd07b91bf4c (127.14.184.194:41833)
I20260926 22:44:49.659113 20152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.657951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40875 (local address 127.14.184.195:41693)
0926 22:44:49.658005 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.658008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.658072 (+ 64us) server_negotiation.cc:407] Connection header received
0926 22:44:49.658141 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.658142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.658148 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.658224 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.658901 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.658902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.658903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.658907 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.658924 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.658928 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.658986 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.659031 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.659047 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:44:49.659137 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.657874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:41693 (local address 127.14.184.193:40875)
0926 22:44:49.658009 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.658020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.658083 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.658229 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.658229 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.658246 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.658802 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.658802 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.658808 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.658819 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.658984 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.658985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.658985 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.659034 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.659047 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260926 22:44:49.659224 20150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.657963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:54315 (local address 127.14.184.194:41833)
0926 22:44:49.658014 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.658016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.658107 (+ 91us) server_negotiation.cc:407] Connection header received
0926 22:44:49.658171 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.658172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.658178 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.658253 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.659001 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.659002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.659003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.659007 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.659029 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.659034 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.659097 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.659146 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.659160 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 22:44:49.659240 20163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.657878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:41833 (local address 127.14.184.193:54315)
0926 22:44:49.658025 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.658035 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.658112 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.658258 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.658258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.658263 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:49.658886 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:44:49.658886 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:49.658894 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:44:49.658918 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:49.659094 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:49.659095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:49.659095 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 22:44:49.659147 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:44:49.659160 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"spinlock_wait_cycles":12544,"thread_start_us":89,"threads_started":1}
I20260926 22:44:49.659574 19962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc1bae05cbf4d1486d79d48bd7c47e9" candidate_uuid: "febf63e62333478bbabb13c37fdb00af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" is_pre_election: true
I20260926 22:44:49.659618 20093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc1bae05cbf4d1486d79d48bd7c47e9" candidate_uuid: "febf63e62333478bbabb13c37fdb00af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" is_pre_election: true
I20260926 22:44:49.659732 20093 raft_consensus.cc:2454] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febf63e62333478bbabb13c37fdb00af in term 0.
I20260926 22:44:49.659732 19962 raft_consensus.cc:2454] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate febf63e62333478bbabb13c37fdb00af in term 0.
I20260926 22:44:49.659948 19764 leader_election.cc:302] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56100b08e5f34f23a0cd17d1f3eb5536, febf63e62333478bbabb13c37fdb00af; no voters:
I20260926 22:44:49.660074 20159 raft_consensus.cc:2781] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:49.660130 20159 raft_consensus.cc:482] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:49.660152 20159 raft_consensus.cc:3037] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:49.660851 20159 raft_consensus.cc:504] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.660986 20159 leader_election.cc:288] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [CANDIDATE]: Term 1 election: Requested vote from peers 56100b08e5f34f23a0cd17d1f3eb5536 (127.14.184.195:41693), 11a4c19f626b49b897f1ddd07b91bf4c (127.14.184.194:41833)
I20260926 22:44:49.661140 20093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc1bae05cbf4d1486d79d48bd7c47e9" candidate_uuid: "febf63e62333478bbabb13c37fdb00af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56100b08e5f34f23a0cd17d1f3eb5536"
I20260926 22:44:49.661176 19962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc1bae05cbf4d1486d79d48bd7c47e9" candidate_uuid: "febf63e62333478bbabb13c37fdb00af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11a4c19f626b49b897f1ddd07b91bf4c"
I20260926 22:44:49.661239 20093 raft_consensus.cc:3037] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:49.661243 19962 raft_consensus.cc:3037] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:49.661942 19962 raft_consensus.cc:2454] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febf63e62333478bbabb13c37fdb00af in term 1.
I20260926 22:44:49.662025 20093 raft_consensus.cc:2454] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate febf63e62333478bbabb13c37fdb00af in term 1.
I20260926 22:44:49.662109 19766 leader_election.cc:302] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11a4c19f626b49b897f1ddd07b91bf4c, febf63e62333478bbabb13c37fdb00af; no voters:
I20260926 22:44:49.662187 20159 raft_consensus.cc:2781] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:49.662323 20159 raft_consensus.cc:686] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [term 1 LEADER]: Becoming Leader. State: Replica: febf63e62333478bbabb13c37fdb00af, State: Running, Role: LEADER
I20260926 22:44:49.662433 20159 consensus_queue.cc:237] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } }
I20260926 22:44:49.663246 19688 catalog_manager.cc:5495] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af reported cstate change: term changed from 0 to 1, leader changed from <none> to febf63e62333478bbabb13c37fdb00af (127.14.184.193). New cstate: current_term: 1 leader_uuid: "febf63e62333478bbabb13c37fdb00af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "febf63e62333478bbabb13c37fdb00af" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:49.676515 20151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.675570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.14.184.193:44047)
0926 22:44:49.675628 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.675630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.675637 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:49.675668 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.675669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.675680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.675733 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.676285 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.676287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.676287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.676291 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.676305 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.676309 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.676356 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.676398 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.676409 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 22:44:49.678228 19962 raft_consensus.cc:1261] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 1 FOLLOWER]: Refusing update from remote peer febf63e62333478bbabb13c37fdb00af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:49.678267 20093 raft_consensus.cc:1261] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 1 FOLLOWER]: Refusing update from remote peer febf63e62333478bbabb13c37fdb00af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:49.678561 20159 consensus_queue.cc:1035] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [LEADER]: Connected to new peer: Peer: permanent_uuid: "11a4c19f626b49b897f1ddd07b91bf4c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:49.678725 20164 consensus_queue.cc:1035] T ecc1bae05cbf4d1486d79d48bd7c47e9 P febf63e62333478bbabb13c37fdb00af [LEADER]: Connected to new peer: Peer: permanent_uuid: "56100b08e5f34f23a0cd17d1f3eb5536" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:49.681223 20167 mvcc.cc:205] Tried to move back new op lower bound from 7333735176919576576 to 7333735176857923584. Current Snapshot: MvccSnapshot[applied={T|T < 7333735176919576576}]
I20260926 22:44:49.682354 20168 mvcc.cc:205] Tried to move back new op lower bound from 7333735176919576576 to 7333735176857923584. Current Snapshot: MvccSnapshot[applied={T|T < 7333735176919576576}]
I20260926 22:44:49.683630 19687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:49.683723 19687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:49.684432 19687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba77be74a0f349088ef81223d50e20fa: Sending DeleteTablet for 3 replicas of tablet ecc1bae05cbf4d1486d79d48bd7c47e9
I20260926 22:44:49.684731 19811 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc1bae05cbf4d1486d79d48bd7c47e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:49 UTC) from {username='slave'} at 127.0.0.1:34118
I20260926 22:44:49.684780 20073 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc1bae05cbf4d1486d79d48bd7c47e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:49 UTC) from {username='slave'} at 127.0.0.1:47098
I20260926 22:44:49.684980 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19749
I20260926 22:44:49.685491 20178 tablet_replica.cc:330] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: stopping tablet replica
I20260926 22:44:49.685585 20178 raft_consensus.cc:2229] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:49.685654 20178 raft_consensus.cc:2258] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:49.685729 19942 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc1bae05cbf4d1486d79d48bd7c47e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:49 UTC) from {username='slave'} at 127.0.0.1:36396
I20260926 22:44:49.685956 20179 tablet_replica.cc:330] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: stopping tablet replica
I20260926 22:44:49.686072 20178 ts_tablet_manager.cc:1905] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:49.686082 20179 raft_consensus.cc:2229] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:49.686179 20179 raft_consensus.cc:2258] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:49.686522 20179 ts_tablet_manager.cc:1905] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:49.688242 20178 ts_tablet_manager.cc:1918] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:49.688325 20178 log.cc:1196] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/ecc1bae05cbf4d1486d79d48bd7c47e9
I20260926 22:44:49.688356 20179 ts_tablet_manager.cc:1918] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:49.688427 20179 log.cc:1196] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/ecc1bae05cbf4d1486d79d48bd7c47e9
I20260926 22:44:49.688597 20178 ts_tablet_manager.cc:1939] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 56100b08e5f34f23a0cd17d1f3eb5536: Deleting consensus metadata
I20260926 22:44:49.688689 20179 ts_tablet_manager.cc:1939] T ecc1bae05cbf4d1486d79d48bd7c47e9 P 11a4c19f626b49b897f1ddd07b91bf4c: Deleting consensus metadata
I20260926 22:44:49.689015 19673 catalog_manager.cc:4841] TS 56100b08e5f34f23a0cd17d1f3eb5536 (127.14.184.195:41693): tablet ecc1bae05cbf4d1486d79d48bd7c47e9 (table test-table [id=88dd45d7553b44ab9880c9f53c7ff259]) successfully deleted
I20260926 22:44:49.689339 19675 catalog_manager.cc:4841] TS 11a4c19f626b49b897f1ddd07b91bf4c (127.14.184.194:41833): tablet ecc1bae05cbf4d1486d79d48bd7c47e9 (table test-table [id=88dd45d7553b44ab9880c9f53c7ff259]) successfully deleted
W20260926 22:44:49.691807 19676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:44047 (error 108)
I20260926 22:44:49.692071 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19880
I20260926 22:44:49.692263 20148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.692148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44047 (local address 127.0.0.1:34134)
0926 22:44:49.692193 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.692226 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:44047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:44:49.692360 19676 catalog_manager.cc:4584] TS febf63e62333478bbabb13c37fdb00af (127.14.184.193:44047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ecc1bae05cbf4d1486d79d48bd7c47e9: Network error: Client connection negotiation failed: client connection to 127.14.184.193:44047: connect: Connection refused (error 111)
I20260926 22:44:49.696866 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20011
I20260926 22:44:49.701964 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 19658
2026-09-26T22:44:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (887 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 22:44:49.721053 15075 mini_oidc.cc:152] Starting JWKS server
I20260926 22:44:49.722049 15075 webserver.cc:460] Webserver started at https://127.0.0.1:43187/ using document root <none> and password file <none>
I20260926 22:44:49.722127 15075 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 22:44:49.722280 15075 webserver.cc:460] Webserver started at http://127.0.0.1:46663/ using document root <none> and password file <none>
2026-09-26T22:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:49Z Disabled control of system clock
I20260926 22:44:49.729924 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38371
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:35397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:38371
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43187/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/testchainca.cert with env {}
W20260926 22:44:49.809716 20186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:49.809938 20186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:49.809973 20186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:49.811103 20186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:49.811165 20186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:49.811184 20186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:49.811205 20186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:49.811285 20186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 22:44:49.811326 20186 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 22:44:49.811350 20186 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43187/jwks.json
I20260926 22:44:49.812879 20186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:38371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43187/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:49.813144 20186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:49.813365 20186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:49.818184 20191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:49.818195 20194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.818292 20186 server_base.cc:1034] running on GCE node
W20260926 22:44:49.818195 20192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:49.818702 20186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:49.819023 20186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:49.820168 20186 hybrid_clock.cc:648] HybridClock initialized: now 1790462689820153 us; error 30 us; skew 500 ppm
I20260926 22:44:49.821465 20186 webserver.cc:460] Webserver started at http://127.14.184.254:33955/ using document root <none> and password file <none>
I20260926 22:44:49.821729 20186 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:49.821811 20186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:49.821960 20186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:49.823110 20186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "20b4cf254fb04fb4bd7a5aba028c16cf"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "71bcd51b1410be0046b82ffca04826c3"
server_key_iv: "767a4e2882580af8a0674005e39b650a"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.823477 20186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "20b4cf254fb04fb4bd7a5aba028c16cf"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "71bcd51b1410be0046b82ffca04826c3"
server_key_iv: "767a4e2882580af8a0674005e39b650a"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.824753 20186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:49.825624 20200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:49.825802 20186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:49.825910 20186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "20b4cf254fb04fb4bd7a5aba028c16cf"
format_stamp: "Formatted at 2026-09-26 22:44:49 on dist-test-slave-3986"
server_key: "71bcd51b1410be0046b82ffca04826c3"
server_key_iv: "767a4e2882580af8a0674005e39b650a"
server_key_version: "encryptionkey@0"
I20260926 22:44:49.825995 20186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:49.842136 20186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:49.842440 20186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:49.842587 20186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:49.846252 20186 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:38371
I20260926 22:44:49.846299 20252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:38371 every 8 connection(s)
I20260926 22:44:49.846728 20186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:49.847790 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:49.849987 20253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf: Bootstrap starting.
I20260926 22:44:49.850570 20253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:49.850889 20253 log.cc:824] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:49.851554 20253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf: No bootstrap required, opened a new log
I20260926 22:44:49.853108 20253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38371 } }
I20260926 22:44:49.853240 20253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:49.853262 20253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b4cf254fb04fb4bd7a5aba028c16cf, State: Initialized, Role: FOLLOWER
I20260926 22:44:49.853366 20253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38371 } }
I20260926 22:44:49.853446 20253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:49.853475 20253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:49.853520 20253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:49.853969 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20186
I20260926 22:44:49.854086 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:49.854051 20253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38371 } }
I20260926 22:44:49.854188 20253 leader_election.cc:302] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20b4cf254fb04fb4bd7a5aba028c16cf; no voters:
I20260926 22:44:49.854298 15075 external_mini_cluster.cc:1428] Setting key 5b96ff313e3a942a6c9205d68a620ce9
I20260926 22:44:49.854396 20253 leader_election.cc:288] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:49.854442 20256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:49.854565 20256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [term 1 LEADER]: Becoming Leader. State: Replica: 20b4cf254fb04fb4bd7a5aba028c16cf, State: Running, Role: LEADER
I20260926 22:44:49.854656 20253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:49.854656 20256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38371 } }
I20260926 22:44:49.855166 20256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20b4cf254fb04fb4bd7a5aba028c16cf. Latest consensus state: current_term: 1 leader_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38371 } } }
I20260926 22:44:49.855255 20256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:49.855396 20258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b4cf254fb04fb4bd7a5aba028c16cf" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38371 } } }
I20260926 22:44:49.855458 20267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:49.856014 20267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:49.856866 20258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:49.859929 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.854895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55828 (local address 127.14.184.254:38371)
0926 22:44:49.855077 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.855085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.855267 (+ 182us) server_negotiation.cc:407] Connection header received
0926 22:44:49.855471 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.855478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.855550 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.855677 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:49.856009 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:49.857281 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:49.858067 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:49.858494 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:49.858629 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:49.859316 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:49.859319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:49.859320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:49.859491 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:49.859523 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:49.859533 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:49.859604 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:49.859677 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 22:44:49.859749 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260926 22:44:49.860952 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:44:49.863622 20267 catalog_manager.cc:1347] Generated new cluster ID: 8decc00c7acf4103825890e33e78ea9e
I20260926 22:44:49.863679 20267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:49.868896 20267 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:49.869356 20267 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:49.878533 20267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20b4cf254fb04fb4bd7a5aba028c16cf: Generated new TSK 0
I20260926 22:44:49.878726 20267 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 22:44:49.911916 20283 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 22:44:49.912019 20283 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 22:44:49.912276 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:49.889263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.14.184.254:38371)
0926 22:44:49.889313 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:49.889316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:49.889423 (+ 107us) server_negotiation.cc:407] Connection header received
0926 22:44:49.889518 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:49.889519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:49.889529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:49.889580 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0926 22:44:49.889730 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:49.890230 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:49.891380 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:49.891557 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:49.891609 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:49.891820 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 22:44:49.911876 (+ 20056us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 22:44:49.912213 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55842: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 22:44:49.912341 20260 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55842: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 22:44:49.912506 20282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:49.889166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38371 (local address 127.0.0.1:55842)
0926 22:44:49.889348 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:49.889361 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:49.889431 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:49.889595 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:49.889596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:49.889598 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 22:44:49.889673 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:49.889675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:49.890304 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:49.890305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:49.891303 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:49.891306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:49.891629 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:49.891630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:49.891703 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:49.891723 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 22:44:49.912029 (+ 20306us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 22:44:49.912139 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.184.254:38371: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.184.254:38371: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43187/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 22:44:49.912906 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20186
2026-09-26T22:44:49Z chronyd exiting
I20260926 22:44:49.948719 15075 test_util.cc:175] -----------------------------------------------
I20260926 22:44:49.948791 15075 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790462668484683-15075-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 22:44:49.949553 15075 mini_oidc.cc:152] Starting JWKS server
I20260926 22:44:49.950237 15075 webserver.cc:460] Webserver started at https://127.0.0.1:39791/ using document root <none> and password file <none>
I20260926 22:44:49.950277 15075 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 22:44:49.950505 15075 webserver.cc:460] Webserver started at http://127.0.0.1:39569/ using document root <none> and password file <none>
2026-09-26T22:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:49Z Disabled control of system clock
I20260926 22:44:49.957810 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44767
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:44767
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39791/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/testchainca.cert with env {}
W20260926 22:44:50.034818 20289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:50.035048 20289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:50.035084 20289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:50.036222 20289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:50.036293 20289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:50.036314 20289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:50.036336 20289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:50.036373 20289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 22:44:50.036398 20289 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 22:44:50.036424 20289 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39791/jwks.json
I20260926 22:44:50.037897 20289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:44767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39791/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:50.038158 20289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:50.038374 20289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:50.043193 20294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:50.043277 20289 server_base.cc:1034] running on GCE node
W20260926 22:44:50.043187 20295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:50.043188 20297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:50.043653 20289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:50.043942 20289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:50.045089 20289 hybrid_clock.cc:648] HybridClock initialized: now 1790462690045075 us; error 29 us; skew 500 ppm
I20260926 22:44:50.046176 20289 webserver.cc:460] Webserver started at http://127.14.184.254:44979/ using document root <none> and password file <none>
I20260926 22:44:50.046379 20289 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:50.046422 20289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:50.046504 20289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:50.047510 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "4069566c040c42318e2055999ffb5de0"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "aabba536cb58e24c21c830e93cf7afe4"
server_key_iv: "7747d18c5b02596eb966835251f9bead"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.047803 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "4069566c040c42318e2055999ffb5de0"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "aabba536cb58e24c21c830e93cf7afe4"
server_key_iv: "7747d18c5b02596eb966835251f9bead"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.049043 20289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:44:50.049685 20303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:50.049839 20289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:50.049906 20289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "4069566c040c42318e2055999ffb5de0"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "aabba536cb58e24c21c830e93cf7afe4"
server_key_iv: "7747d18c5b02596eb966835251f9bead"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.049953 20289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:50.062564 20289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:50.062862 20289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:50.062968 20289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:50.066417 20289 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:44767
I20260926 22:44:50.066481 20355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:44767 every 8 connection(s)
I20260926 22:44:50.066825 20289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:50.067487 20356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:50.070171 20356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0: Bootstrap starting.
I20260926 22:44:50.070761 20356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:50.071051 20356 log.cc:824] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:50.071640 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20289
I20260926 22:44:50.071754 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:50.071797 20356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0: No bootstrap required, opened a new log
I20260926 22:44:50.071970 15075 external_mini_cluster.cc:1428] Setting key 80918f1ce172c8660be21ac316dd85ce
I20260926 22:44:50.074261 20356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4069566c040c42318e2055999ffb5de0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44767 } }
I20260926 22:44:50.074429 20356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:50.074467 20356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4069566c040c42318e2055999ffb5de0, State: Initialized, Role: FOLLOWER
I20260926 22:44:50.074592 20356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4069566c040c42318e2055999ffb5de0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44767 } }
I20260926 22:44:50.074703 20356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:50.074765 20356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:50.074820 20356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:50.075462 20356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4069566c040c42318e2055999ffb5de0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44767 } }
I20260926 22:44:50.075587 20356 leader_election.cc:302] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4069566c040c42318e2055999ffb5de0; no voters:
I20260926 22:44:50.075780 20356 leader_election.cc:288] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:50.075798 20361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:50.075906 20361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [term 1 LEADER]: Becoming Leader. State: Replica: 4069566c040c42318e2055999ffb5de0, State: Running, Role: LEADER
I20260926 22:44:50.076038 20356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:50.076042 20361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4069566c040c42318e2055999ffb5de0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44767 } }
I20260926 22:44:50.076442 20361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4069566c040c42318e2055999ffb5de0. Latest consensus state: current_term: 1 leader_uuid: "4069566c040c42318e2055999ffb5de0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4069566c040c42318e2055999ffb5de0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44767 } } }
I20260926 22:44:50.076450 20362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4069566c040c42318e2055999ffb5de0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4069566c040c42318e2055999ffb5de0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44767 } } }
I20260926 22:44:50.076606 20361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:50.076651 20362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:50.076911 20370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:50.077411 20370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:50.077574 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:50.072441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.14.184.254:44767)
0926 22:44:50.072653 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:50.072660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:50.072683 (+ 23us) server_negotiation.cc:407] Connection header received
0926 22:44:50.073135 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:50.073141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:50.073207 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:50.073316 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:50.073529 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.074315 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.075444 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.076020 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.076132 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.076807 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:50.076810 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:50.076811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:50.077021 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:50.077060 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:50.077074 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:50.077135 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:50.077241 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 22:44:50.077320 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260926 22:44:50.078807 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:44:50.079231 20370 catalog_manager.cc:1347] Generated new cluster ID: da46be5fcce540c3b4d4e107c25ce544
I20260926 22:44:50.079272 20370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:50.085624 20370 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:50.086094 20370 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:50.092317 20370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4069566c040c42318e2055999ffb5de0: Generated new TSK 0
I20260926 22:44:50.092487 20370 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 22:44:50.114634 20386 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 22:44:50.114785 20386 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 22:44:50.115068 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:50.100905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.14.184.254:44767)
0926 22:44:50.100972 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:50.100973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:50.101001 (+ 28us) server_negotiation.cc:407] Connection header received
0926 22:44:50.101062 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:50.101063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:50.101077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:50.101160 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0926 22:44:50.101295 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.102079 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.102863 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.103189 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.103259 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.103394 (+ 135us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 22:44:50.114856 (+ 11462us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 22:44:50.114998 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 22:44:50.115096 20385 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:50.100771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44767 (local address 127.0.0.1:55358)
0926 22:44:50.100941 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:50.100951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:50.101006 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:50.101164 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:50.101165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:50.101167 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 22:44:50.101227 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:50.101228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.102151 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.102151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:50.102785 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:50.102788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.103265 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.103266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:50.103309 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.103313 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 22:44:50.114954 (+ 11641us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 22:44:50.115034 (+ 80us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.184.254:44767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
W20260926 22:44:50.115128 20360 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260926 22:44:50.115389 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20289
2026-09-26T22:44:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 22:44:50.152768 15075 mini_oidc.cc:152] Starting JWKS server
I20260926 22:44:50.153467 15075 webserver.cc:460] Webserver started at https://127.0.0.1:40525/ using document root <none> and password file <none>
I20260926 22:44:50.153537 15075 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 22:44:50.153717 15075 webserver.cc:460] Webserver started at http://127.0.0.1:41939/ using document root <none> and password file <none>
2026-09-26T22:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:50Z Disabled control of system clock
I20260926 22:44:50.161214 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33565
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:33565
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40525/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 22:44:50.239197 20392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:50.239390 20392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:50.239411 20392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:50.240558 20392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:50.240599 20392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:50.240612 20392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:50.240624 20392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:50.240636 20392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 22:44:50.240648 20392 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 22:44:50.240659 20392 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40525/jwks.json
I20260926 22:44:50.241950 20392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:33565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40525/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:50.242159 20392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:50.242441 20392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:50.247068 20400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:50.247076 20392 server_base.cc:1034] running on GCE node
W20260926 22:44:50.247211 20397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:50.247207 20398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:50.247582 20392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:50.247852 20392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:50.248997 20392 hybrid_clock.cc:648] HybridClock initialized: now 1790462690248978 us; error 35 us; skew 500 ppm
I20260926 22:44:50.250219 20392 webserver.cc:460] Webserver started at http://127.14.184.254:40415/ using document root <none> and password file <none>
I20260926 22:44:50.250458 20392 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:50.250504 20392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:50.250655 20392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:50.251698 20392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "c1d538125c2a437ea396636325081957"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "90a7e80fae9e6ab702e44e93625c800a"
server_key_iv: "2c80929ce49a0415e36c7be240f071b4"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.252053 20392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "c1d538125c2a437ea396636325081957"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "90a7e80fae9e6ab702e44e93625c800a"
server_key_iv: "2c80929ce49a0415e36c7be240f071b4"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.253281 20392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:44:50.253919 20406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:50.254093 20392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:50.254163 20392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "c1d538125c2a437ea396636325081957"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "90a7e80fae9e6ab702e44e93625c800a"
server_key_iv: "2c80929ce49a0415e36c7be240f071b4"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.254212 20392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:50.270164 20392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:50.270404 20392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:50.270502 20392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:50.274359 20392 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:33565
I20260926 22:44:50.274389 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:33565 every 8 connection(s)
I20260926 22:44:50.274755 20392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:50.275465 20459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:50.275472 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20392
I20260926 22:44:50.275583 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:50.275763 15075 external_mini_cluster.cc:1428] Setting key ba8dc22584b4409d28ce64b94876aa20
I20260926 22:44:50.279184 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957: Bootstrap starting.
I20260926 22:44:50.279822 20459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:50.280154 20459 log.cc:824] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:50.280979 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957: No bootstrap required, opened a new log
I20260926 22:44:50.281565 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:50.276290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.14.184.254:33565)
0926 22:44:50.276472 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:50.276479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:50.276503 (+ 24us) server_negotiation.cc:407] Connection header received
0926 22:44:50.277130 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:50.277137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:50.277196 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:50.277310 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:50.277561 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.278391 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.279531 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.279998 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.280092 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.280880 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:50.280883 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:50.280883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:50.281085 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:50.281125 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:50.281137 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:50.281187 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:50.281269 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 22:44:50.281358 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260926 22:44:50.282511 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:44:50.283390 20459 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d538125c2a437ea396636325081957" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33565 } }
I20260926 22:44:50.283552 20459 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:50.283578 20459 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d538125c2a437ea396636325081957, State: Initialized, Role: FOLLOWER
I20260926 22:44:50.283718 20459 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d538125c2a437ea396636325081957" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33565 } }
I20260926 22:44:50.283825 20459 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:50.283865 20459 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:50.283917 20459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:50.284458 20459 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d538125c2a437ea396636325081957" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33565 } }
I20260926 22:44:50.284569 20459 leader_election.cc:302] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1d538125c2a437ea396636325081957; no voters:
I20260926 22:44:50.284718 20459 leader_election.cc:288] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:50.284749 20464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:50.284926 20464 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [term 1 LEADER]: Becoming Leader. State: Replica: c1d538125c2a437ea396636325081957, State: Running, Role: LEADER
I20260926 22:44:50.284987 20459 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:50.285104 20464 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d538125c2a437ea396636325081957" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33565 } }
I20260926 22:44:50.285524 20466 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1d538125c2a437ea396636325081957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d538125c2a437ea396636325081957" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33565 } } }
I20260926 22:44:50.285524 20465 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1d538125c2a437ea396636325081957. Latest consensus state: current_term: 1 leader_uuid: "c1d538125c2a437ea396636325081957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d538125c2a437ea396636325081957" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33565 } } }
I20260926 22:44:50.285681 20466 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:50.285691 20465 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957 [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:50.286104 20473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:50.286562 20473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:50.288030 20473 catalog_manager.cc:1347] Generated new cluster ID: 0deefca1d31642dcb3ece87ab879d06a
I20260926 22:44:50.288105 20473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:50.302516 20473 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:50.303072 20473 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:50.305016 20473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1d538125c2a437ea396636325081957: Generated new TSK 0
I20260926 22:44:50.305193 20473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:44:50.314944 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:50.311326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.14.184.254:33565)
0926 22:44:50.311392 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:50.311395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:50.311529 (+ 134us) server_negotiation.cc:407] Connection header received
0926 22:44:50.311598 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:50.311599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:50.311609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:50.311665 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0926 22:44:50.311791 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.312285 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.313407 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.313582 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.313629 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.313818 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 22:44:50.314723 (+ 905us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 22:44:50.314872 (+ 149us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38870: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 22:44:50.314994 20462 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38870: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 22:44:50.315045 20488 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:50.311273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33565 (local address 127.0.0.1:38870)
0926 22:44:50.311453 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:50.311464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:50.311532 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:50.311670 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:50.311671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:50.311674 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0926 22:44:50.311733 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:50.311735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.312383 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.312384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:50.313322 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:50.313324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.313665 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.313665 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:50.313727 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.313733 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 22:44:50.314832 (+ 1099us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 22:44:50.314939 (+ 107us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.184.254:33565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40525/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260926 22:44:50.315254 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20392
E20260926 22:44:50.325371 20489 webserver.cc:533] Webserver: sslize failed:
E20260926 22:44:50.325452 20489 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T22:44:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (200 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T22:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:50Z Disabled control of system clock
I20260926 22:44:50.360301 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43129
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43129
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/testchainca.cert with env {}
W20260926 22:44:50.438117 20493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:50.438310 20493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:50.438331 20493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:50.439528 20493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:50.439569 20493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:50.439582 20493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:50.439620 20493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:50.439651 20493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 22:44:50.439671 20493 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 22:44:50.439700 20493 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 22:44:50.441167 20493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:50.441370 20493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:50.441565 20493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:50.446233 20498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:50.446211 20499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:50.446242 20493 server_base.cc:1034] running on GCE node
W20260926 22:44:50.446199 20501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:50.446658 20493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:50.446962 20493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:50.448127 20493 hybrid_clock.cc:648] HybridClock initialized: now 1790462690448109 us; error 31 us; skew 500 ppm
I20260926 22:44:50.449347 20493 webserver.cc:460] Webserver started at http://127.14.184.254:43735/ using document root <none> and password file <none>
I20260926 22:44:50.449599 20493 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:50.449667 20493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:50.449815 20493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:50.450909 20493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "a6e2196b306c4750856df1b55e705f1a"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "72bd0d952b0df9dbc45b25e4de1daca5"
server_key_iv: "d76c6236cf0daac8f39ca178c18ab85a"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.451256 20493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "a6e2196b306c4750856df1b55e705f1a"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "72bd0d952b0df9dbc45b25e4de1daca5"
server_key_iv: "d76c6236cf0daac8f39ca178c18ab85a"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.452510 20493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:50.453181 20507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:50.453343 20493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:50.453450 20493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "a6e2196b306c4750856df1b55e705f1a"
format_stamp: "Formatted at 2026-09-26 22:44:50 on dist-test-slave-3986"
server_key: "72bd0d952b0df9dbc45b25e4de1daca5"
server_key_iv: "d76c6236cf0daac8f39ca178c18ab85a"
server_key_version: "encryptionkey@0"
I20260926 22:44:50.453521 20493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:50.488448 20493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:50.488781 20493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:50.488938 20493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:50.492748 20493 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43129
I20260926 22:44:50.492785 20559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43129 every 8 connection(s)
I20260926 22:44:50.493156 20493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:50.494114 20560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:50.494417 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20493
I20260926 22:44:50.494534 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:50.494784 15075 external_mini_cluster.cc:1428] Setting key 589727bf0127d3f1ee710fcef437868f
I20260926 22:44:50.497287 20560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a: Bootstrap starting.
I20260926 22:44:50.497944 20560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:50.498219 20560 log.cc:824] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:50.498996 20560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a: No bootstrap required, opened a new log
I20260926 22:44:50.500957 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:50.495319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40904 (local address 127.14.184.254:43129)
0926 22:44:50.495553 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:50.495564 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:50.495597 (+ 33us) server_negotiation.cc:407] Connection header received
0926 22:44:50.496150 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:50.496159 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:50.496236 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:50.496397 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:50.496553 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.497915 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.498914 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.499510 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.499632 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.500247 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:50.500251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:50.500251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:44:50.500460 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:50.500501 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:44:50.500515 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:50.500591 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:50.500628 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 22:44:50.500712 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260926 22:44:50.501312 20560 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e2196b306c4750856df1b55e705f1a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43129 } }
I20260926 22:44:50.501473 20560 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:50.501506 20560 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6e2196b306c4750856df1b55e705f1a, State: Initialized, Role: FOLLOWER
I20260926 22:44:50.501622 20560 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e2196b306c4750856df1b55e705f1a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43129 } }
I20260926 22:44:50.501703 20560 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:50.501735 20560 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:50.501784 20560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:50.501945 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:44:50.502434 20560 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e2196b306c4750856df1b55e705f1a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43129 } }
I20260926 22:44:50.502547 20560 leader_election.cc:302] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6e2196b306c4750856df1b55e705f1a; no voters:
I20260926 22:44:50.502715 20560 leader_election.cc:288] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:50.502743 20565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:50.502967 20560 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:50.503028 20565 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [term 1 LEADER]: Becoming Leader. State: Replica: a6e2196b306c4750856df1b55e705f1a, State: Running, Role: LEADER
I20260926 22:44:50.503110 20565 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e2196b306c4750856df1b55e705f1a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43129 } }
I20260926 22:44:50.503472 20565 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6e2196b306c4750856df1b55e705f1a. Latest consensus state: current_term: 1 leader_uuid: "a6e2196b306c4750856df1b55e705f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e2196b306c4750856df1b55e705f1a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43129 } } }
I20260926 22:44:50.503505 20566 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6e2196b306c4750856df1b55e705f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e2196b306c4750856df1b55e705f1a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43129 } } }
I20260926 22:44:50.503563 20565 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:50.503624 20566 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:50.503880 20574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:50.504410 20574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:50.505913 20574 catalog_manager.cc:1347] Generated new cluster ID: 4dcff275b6df461cbd36df88f1c6b24c
I20260926 22:44:50.505972 20574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:50.518416 20574 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:50.518993 20574 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:50.525396 20574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6e2196b306c4750856df1b55e705f1a: Generated new TSK 0
I20260926 22:44:50.525578 20574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:44:50.968158 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:50.531803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.14.184.254:43129)
0926 22:44:50.531857 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:50.531859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:50.531945 (+ 86us) server_negotiation.cc:407] Connection header received
0926 22:44:50.532022 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:50.532023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:50.532035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:50.532106 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0926 22:44:50.532224 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.533013 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.533752 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.534073 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.534144 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.534288 (+ 144us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 22:44:50.967947 (+433659us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 22:44:50.968097 (+ 150us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40918: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260926 22:44:50.968219 20563 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40918: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 22:44:50.968338 20589 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:50.531728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43129 (local address 127.0.0.1:40918)
0926 22:44:50.531871 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:50.531883 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:50.531950 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:50.532111 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:50.532112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:50.532114 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 22:44:50.532164 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:50.532166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.533084 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.533085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:50.533672 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:50.533673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:50.534154 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:50.534155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:50.534194 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:50.534199 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 22:44:50.968081 (+433882us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 22:44:50.968198 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.184.254:43129: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260926 22:44:50.968847 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20493
2026-09-26T22:44:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (633 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:51 dist-test-slave-3986 krb5kdc[20601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:51 dist-test-slave-3986 krb5kdc[20601](info): set up 2 sockets
Sep 26 22:44:51 dist-test-slave-3986 krb5kdc[20601](info): commencing operation
krb5kdc: starting...
W20260926 22:44:52.175248 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:44:52.287822 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43999
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:52.366389 20614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:52.366611 20614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:52.366645 20614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:52.368278 20614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:52.368335 20614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:52.368377 20614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:52.368396 20614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:52.368418 20614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:52.370846 20614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:52.371094 20614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:52.371343 20614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:52.377148 20619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.377158 20620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.378294 20622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:52.382050 20614 server_base.cc:1034] running on GCE node
I20260926 22:44:52.382333 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:52.382647 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:52.383836 20614 hybrid_clock.cc:648] HybridClock initialized: now 1790462692383819 us; error 47 us; skew 500 ppm
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.391618 20614 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:44:52.392087 20614 webserver.cc:460] Webserver started at http://127.14.184.254:38199/ using document root <none> and password file <none>
I20260926 22:44:52.392313 20614 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:52.392388 20614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:52.392539 20614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:52.393576 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "f3642358ea154de9adcc4c1ed03834be"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "00d643bb7faa9fc186627f0feea25e3a"
server_key_iv: "693d44482c8a17cd74117c2047255dd9"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.393949 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "f3642358ea154de9adcc4c1ed03834be"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "00d643bb7faa9fc186627f0feea25e3a"
server_key_iv: "693d44482c8a17cd74117c2047255dd9"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.395332 20614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:52.396039 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.396209 20614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:52.396313 20614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "f3642358ea154de9adcc4c1ed03834be"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "00d643bb7faa9fc186627f0feea25e3a"
server_key_iv: "693d44482c8a17cd74117c2047255dd9"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.396399 20614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:52.407709 20614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:52.409885 20614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:52.410038 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:52.413962 20614 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43999
I20260926 22:44:52.413995 20681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43999 every 8 connection(s)
I20260926 22:44:52.414323 20614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:52.415170 20682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:52.417258 20682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be: Bootstrap starting.
I20260926 22:44:52.417748 20682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:52.417989 20682 log.cc:824] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:52.418574 20682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be: No bootstrap required, opened a new log
I20260926 22:44:52.420148 20682 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3642358ea154de9adcc4c1ed03834be" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43999 } }
I20260926 22:44:52.420275 20682 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:52.420297 20682 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3642358ea154de9adcc4c1ed03834be, State: Initialized, Role: FOLLOWER
I20260926 22:44:52.420423 20682 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3642358ea154de9adcc4c1ed03834be" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43999 } }
I20260926 22:44:52.420501 20682 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:52.420523 20682 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:52.420552 20682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:52.420997 20682 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3642358ea154de9adcc4c1ed03834be" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43999 } }
I20260926 22:44:52.421077 20682 leader_election.cc:302] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3642358ea154de9adcc4c1ed03834be; no voters:
I20260926 22:44:52.421197 20682 leader_election.cc:288] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:52.421257 20685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:52.421414 20682 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:52.421640 20685 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [term 1 LEADER]: Becoming Leader. State: Replica: f3642358ea154de9adcc4c1ed03834be, State: Running, Role: LEADER
I20260926 22:44:52.421748 20685 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3642358ea154de9adcc4c1ed03834be" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43999 } }
I20260926 22:44:52.421932 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20614
I20260926 22:44:52.422058 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:52.422049 20685 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3642358ea154de9adcc4c1ed03834be. Latest consensus state: current_term: 1 leader_uuid: "f3642358ea154de9adcc4c1ed03834be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3642358ea154de9adcc4c1ed03834be" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43999 } } }
I20260926 22:44:52.422075 20686 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3642358ea154de9adcc4c1ed03834be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3642358ea154de9adcc4c1ed03834be" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43999 } } }
I20260926 22:44:52.422119 20685 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:52.422139 20686 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:52.422273 15075 external_mini_cluster.cc:1428] Setting key 2afc69915580b5ebac485525c4887410
I20260926 22:44:52.422639 20694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:52.423394 20694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:52.425946 20694 catalog_manager.cc:1347] Generated new cluster ID: 9c7f2805fb77462c99ff973968e826af
I20260926 22:44:52.425998 20694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:52.433132 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.422918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.14.184.254:43999)
0926 22:44:52.423072 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.423081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.423277 (+ 196us) server_negotiation.cc:407] Connection header received
0926 22:44:52.423442 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.423449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.423502 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.423638 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.424360 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.425236 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.426245 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.426899 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.427004 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.429676 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.429680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.429680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.429692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.431555 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.431943 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.431946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.431946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.432044 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.432257 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.432258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.432258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.432533 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.432697 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.432804 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.432935 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260926 22:44:52.433854 20694 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:52.434340 20694 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:52.443063 20694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3642358ea154de9adcc4c1ed03834be: Generated new TSK 0
I20260926 22:44:52.443214 20694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:44:52.469630 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43999
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:52.548815 20708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:52.549005 20708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:52.549026 20708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:52.550084 20708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:52.550117 20708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:52.550143 20708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:52.551613 20708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:52.551800 20708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:52.551962 20708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:52.556734 20708 server_base.cc:1034] running on GCE node
W20260926 22:44:52.556795 20716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.556707 20713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.556836 20714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:52.557214 20708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:52.557461 20708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:52.558631 20708 hybrid_clock.cc:648] HybridClock initialized: now 1790462692558618 us; error 36 us; skew 500 ppm
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.561017 20708 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:44:52.561455 20708 webserver.cc:460] Webserver started at http://127.14.184.193:44055/ using document root <none> and password file <none>
I20260926 22:44:52.561671 20708 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:52.561713 20708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:52.561874 20708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:52.562902 20708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "bd77f6ab61cf45b98ec6c3c110100ae0"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "549183047fe6e0947827be51bba27a32"
server_key_iv: "32e1418d62ee07fed55e104d3b6e2914"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.563236 20708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "bd77f6ab61cf45b98ec6c3c110100ae0"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "549183047fe6e0947827be51bba27a32"
server_key_iv: "32e1418d62ee07fed55e104d3b6e2914"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.564571 20708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:52.565222 20723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.565409 20708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:52.565510 20708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "bd77f6ab61cf45b98ec6c3c110100ae0"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "549183047fe6e0947827be51bba27a32"
server_key_iv: "32e1418d62ee07fed55e104d3b6e2914"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.565585 20708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:52.581735 20708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:52.583503 20708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:52.583669 20708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:52.583879 20708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:52.584167 20708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:52.584220 20708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.584260 20708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:52.584287 20708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.589903 20708 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:36537
I20260926 22:44:52.589947 20836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:36537 every 8 connection(s)
I20260926 22:44:52.590261 20708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:52.593954 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20708
I20260926 22:44:52.594071 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:52.594273 15075 external_mini_cluster.cc:1428] Setting key 7ebba92e55cccabe520d947b91885018
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:52.599678 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.590945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:38959 (local address 127.14.184.254:43999)
0926 22:44:52.591000 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.591002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.591188 (+ 186us) server_negotiation.cc:407] Connection header received
0926 22:44:52.592058 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.592059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.592071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.592130 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.592691 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.593188 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.594138 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.594462 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.594555 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.596878 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.596879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.596880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.596888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.598282 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.598708 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.598710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.598712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.598771 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.599047 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.599048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.599048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.599160 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.599230 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.599506 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.599592 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:44:52.599864 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.590918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43999 (local address 127.14.184.193:38959)
0926 22:44:52.591065 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:52.591106 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:52.591972 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:52.592183 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:52.592186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:52.592444 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:52.592624 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:52.592628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.593265 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.593265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:52.594028 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:52.594029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.594561 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.594562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:52.594758 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.595245 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:52.595253 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:52.596776 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:52.598375 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:52.598382 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:52.598384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:52.598600 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:52.598835 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:52.598836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:52.598837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:52.598959 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:52.599236 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:52.599242 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:52.599378 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 22:44:52.599483 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 22:44:52.599631 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260926 22:44:52.600315 20837 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43999
I20260926 22:44:52.600404 20837 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:52.600579 20837 heartbeater.cc:511] Master 127.14.184.254:43999 requested a full tablet report, sending...
I20260926 22:44:52.601091 20646 ts_manager.cc:194] Registered new tserver with Master: bd77f6ab61cf45b98ec6c3c110100ae0 (127.14.184.193:36537)
I20260926 22:44:52.601683 20646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:38959
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:44:52.618266 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43999
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:52.696748 20842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:52.696946 20842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:52.696970 20842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:52.698102 20842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:52.698141 20842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:52.698167 20842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:52.699803 20842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:52.700043 20842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:52.700271 20842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:52.705128 20850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.705235 20848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.705295 20847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:52.705288 20842 server_base.cc:1034] running on GCE node
I20260926 22:44:52.705577 20842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:52.705826 20842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:52.706988 20842 hybrid_clock.cc:648] HybridClock initialized: now 1790462692706975 us; error 27 us; skew 500 ppm
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.709842 20842 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:44:52.710386 20842 webserver.cc:460] Webserver started at http://127.14.184.194:45547/ using document root <none> and password file <none>
I20260926 22:44:52.710626 20842 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:52.710713 20842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:52.710868 20842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:52.711930 20842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "34ab59f13b4d4136b05f99e5c80ea2d4"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "921e76008f04db52a71e4468e9ac4423"
server_key_iv: "401764436e813aab644fda39d7cb5ad3"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.712234 20842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "34ab59f13b4d4136b05f99e5c80ea2d4"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "921e76008f04db52a71e4468e9ac4423"
server_key_iv: "401764436e813aab644fda39d7cb5ad3"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.713436 20842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:52.714107 20857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.714257 20842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:52.714324 20842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "34ab59f13b4d4136b05f99e5c80ea2d4"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "921e76008f04db52a71e4468e9ac4423"
server_key_iv: "401764436e813aab644fda39d7cb5ad3"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.714373 20842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:52.724001 20842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:52.725626 20842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:52.725735 20842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:52.725908 20842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:52.726138 20842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:52.726168 20842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.726188 20842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:52.726202 20842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.732155 20842 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:37771
I20260926 22:44:52.732225 20970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:37771 every 8 connection(s)
I20260926 22:44:52.732523 20842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:52.741027 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.733282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:60793 (local address 127.14.184.254:43999)
0926 22:44:52.733325 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.733327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.733493 (+ 166us) server_negotiation.cc:407] Connection header received
0926 22:44:52.734377 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.734378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.734388 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.734437 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.735084 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.735579 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.736496 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.736670 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.736718 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.738567 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.738568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.738568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.738574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.739977 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.740299 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.740300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.740300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.740347 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.740547 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.740547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.740548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.740640 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.740688 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.740903 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.740965 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:44:52.741196 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.733243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43999 (local address 127.14.184.194:60793)
0926 22:44:52.733393 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:52.733427 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:52.734282 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:52.734496 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:52.734499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:52.734824 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:52.735015 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:52.735019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.735655 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.735656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:52.736426 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:52.736428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.736736 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.736737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:52.736827 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.737242 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:52.737250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:52.738486 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:52.740041 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:52.740046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:52.740048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:52.740237 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:52.740409 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:52.740409 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:52.740409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:52.740495 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:52.740692 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:52.740695 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:52.740796 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 22:44:52.740878 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 22:44:52.740984 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260926 22:44:52.741704 20971 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43999
I20260926 22:44:52.741813 20971 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:52.741992 20971 heartbeater.cc:511] Master 127.14.184.254:43999 requested a full tablet report, sending...
I20260926 22:44:52.742380 20646 ts_manager.cc:194] Registered new tserver with Master: 34ab59f13b4d4136b05f99e5c80ea2d4 (127.14.184.194:37771)
I20260926 22:44:52.742583 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20842
I20260926 22:44:52.742702 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:52.742893 15075 external_mini_cluster.cc:1428] Setting key b8345c2aa52ef1788d346e42c3866e09
I20260926 22:44:52.742904 20646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:60793
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:44:52.769696 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43999
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:52.863160 20976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:52.863345 20976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:52.863368 20976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:52.864439 20976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:52.864471 20976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:52.864496 20976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:52.865867 20976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:52.866048 20976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:52.866209 20976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:52.871133 20981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:52.871266 20976 server_base.cc:1034] running on GCE node
W20260926 22:44:52.871142 20982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:52.871320 20984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:52.871591 20976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:52.871845 20976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:52.872987 20976 hybrid_clock.cc:648] HybridClock initialized: now 1790462692872966 us; error 36 us; skew 500 ppm
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.875408 20976 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:44:52.875862 20976 webserver.cc:460] Webserver started at http://127.14.184.195:44625/ using document root <none> and password file <none>
I20260926 22:44:52.876098 20976 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:52.876142 20976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:52.876291 20976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:52.877264 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "e0cc7a0be8f34f1a82c8ee62385e0f8b"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "ec2bffb3c2521bb88228f272a9c40923"
server_key_iv: "98411a247a8b60236b8ffc9530c72c96"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.877600 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "e0cc7a0be8f34f1a82c8ee62385e0f8b"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "ec2bffb3c2521bb88228f272a9c40923"
server_key_iv: "98411a247a8b60236b8ffc9530c72c96"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.878935 20976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:52.879596 20991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.879760 20976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:52.879858 20976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "e0cc7a0be8f34f1a82c8ee62385e0f8b"
format_stamp: "Formatted at 2026-09-26 22:44:52 on dist-test-slave-3986"
server_key: "ec2bffb3c2521bb88228f272a9c40923"
server_key_iv: "98411a247a8b60236b8ffc9530c72c96"
server_key_version: "encryptionkey@0"
I20260926 22:44:52.879937 20976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:52.903887 20976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:52.905622 20976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:52.905781 20976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:52.906014 20976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:52.906309 20976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:52.906363 20976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.906394 20976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:52.906424 20976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:52.912185 20976 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:35485
I20260926 22:44:52.912235 21104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:35485 every 8 connection(s)
I20260926 22:44:52.912531 20976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:52.915951 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 20976
I20260926 22:44:52.916066 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:52.916267 15075 external_mini_cluster.cc:1428] Setting key c601d599e8783192a802d85883ee2309
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:44:52.922078 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.913204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:44091 (local address 127.14.184.254:43999)
0926 22:44:52.913289 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.913292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.913480 (+ 188us) server_negotiation.cc:407] Connection header received
0926 22:44:52.914426 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.914427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.914439 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.914492 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.915179 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.915668 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.917074 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.917300 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.917381 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.919517 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.919518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.919519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.919526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.920649 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.921053 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.921054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.921055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.921111 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.921357 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.921358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.921359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.921517 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.921590 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.921857 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.921972 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 22:44:52.922250 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.913186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43999 (local address 127.14.184.195:44091)
0926 22:44:52.913352 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:52.913399 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:52.914310 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:52.914569 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:52.914574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:52.914904 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:52.915082 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:52.915086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.915763 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.915764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:52.916987 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:52.916989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.917397 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.917398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:52.917531 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.918075 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:52.918083 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:52.919424 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:52.920743 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:52.920748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:52.920750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:52.920995 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:52.921185 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:52.921186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:52.921186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:52.921289 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:52.921592 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:52.921598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:52.921728 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 22:44:52.921832 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 22:44:52.922026 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260926 22:44:52.922735 21105 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43999
I20260926 22:44:52.922842 21105 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:52.923033 21105 heartbeater.cc:511] Master 127.14.184.254:43999 requested a full tablet report, sending...
I20260926 22:44:52.923347 20646 ts_manager.cc:194] Registered new tserver with Master: e0cc7a0be8f34f1a82c8ee62385e0f8b (127.14.184.195:35485)
I20260926 22:44:52.923707 20646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:44091
I20260926 22:44:52.927840 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/test-admin.keytab.
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.941948 15075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 22:44:52.943555 21112 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:52.942755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.14.184.193:36537)
0926 22:44:52.942939 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.942944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.942955 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:52.942980 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.942982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.943020 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.943103 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.943470 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60454: BlockingRecv error: recv got EOF from 127.0.0.1:60454 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
W20260926 22:44:52.943632 15075 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc' of type FILE
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.944553 15075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.945397 15075 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:52.952780 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.945846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.14.184.193:36537)
0926 22:44:52.945905 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.945907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.945943 (+ 36us) server_negotiation.cc:407] Connection header received
0926 22:44:52.946021 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.946021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.946032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.946101 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.946484 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.947253 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.947969 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.948288 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.948376 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.950231 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.950240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.950240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.950247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.951614 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.951981 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.951984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.951984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.952042 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.952239 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.952240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.952241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.952376 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.952522 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.952593 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.952668 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 22:44:52.954516 21112 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:52.953959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.14.184.193:36537)
0926 22:44:52.953994 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.953996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.954004 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:52.954051 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.954052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.954060 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.954132 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.954448 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60466: BlockingRecv error: recv got EOF from 127.0.0.1:60466 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 22:44:52.954655 15075 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc)
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.955619 15075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.956470 15075 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:52.963172 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.956942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.14.184.193:36537)
0926 22:44:52.956993 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.956995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.957061 (+ 66us) server_negotiation.cc:407] Connection header received
0926 22:44:52.957133 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.957134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.957144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.957191 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.957549 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.958020 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.958697 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.958932 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.959018 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.960919 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.960921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.960921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.960928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.962019 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.962343 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.962344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.962344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.962403 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.962620 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.962621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.962621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.962768 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.962866 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.962996 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.963078 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260926 22:44:52.964550 21112 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:52.964104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.14.184.193:36537)
0926 22:44:52.964136 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.964138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.964173 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:44:52.964216 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.964217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.964224 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.964270 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.964503 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60480: BlockingRecv error: recv got EOF from 127.0.0.1:60480 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:44:52.964638 15075 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790462668484683-15075-0/krb5kdc/krb5cc' of type FILE
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.965535 15075 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:52.966342 15075 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 22:44:52 dist-test-slave-3986 krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:44:52.972859 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:52.966874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.14.184.193:36537)
0926 22:44:52.966920 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:52.966921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:52.967011 (+ 90us) server_negotiation.cc:407] Connection header received
0926 22:44:52.967081 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:52.967082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:52.967090 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:52.967132 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:52.967493 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.967929 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.968592 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:52.968781 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:52.968853 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:52.970658 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:52.970659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:52.970659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:52.970682 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:52.971758 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.972130 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.972131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.972132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.972187 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:52.972384 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:52.972385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:52.972386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:52.972478 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:52.972569 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:52.972695 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 22:44:52.972765 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 22:44:52.973438 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20708
I20260926 22:44:52.978077 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20842
I20260926 22:44:52.982419 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20976
I20260926 22:44:52.986582 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 20614
2026-09-26T22:44:52Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1846 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:53 dist-test-slave-3986 krb5kdc[21125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:53 dist-test-slave-3986 krb5kdc[21125](info): set up 2 sockets
Sep 26 22:44:53 dist-test-slave-3986 krb5kdc[21125](info): commencing operation
krb5kdc: starting...
W20260926 22:44:54.031121 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab.
Entry for principal oryx/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab.
I20260926 22:44:54.146286 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43809
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab
--principal=oryx/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:54.225785 21138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:54.225975 21138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:54.225996 21138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:54.227118 21138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:54.227159 21138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:54.227173 21138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:54.227185 21138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:54.227198 21138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:54.228587 21138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab
--principal=oryx/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:54.228781 21138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:54.228936 21138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:54.233578 21143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.233628 21144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.233639 21146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:54.233657 21138 server_base.cc:1034] running on GCE node
I20260926 22:44:54.233975 21138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:54.234258 21138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:54.235395 21138 hybrid_clock.cc:648] HybridClock initialized: now 1790462694235381 us; error 36 us; skew 500 ppm
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:54.237818 21138 init.cc:377] Logged in from keytab as oryx/127.14.184.254@KRBTEST.COM (short username oryx)
I20260926 22:44:54.238279 21138 webserver.cc:460] Webserver started at http://127.14.184.254:35393/ using document root <none> and password file <none>
I20260926 22:44:54.238504 21138 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:54.238570 21138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:54.238754 21138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:54.239761 21138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "c988e7c94b20406e9783b933f740c88e"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "60989374ca7be3262bbed005a231302a"
server_key_iv: "856a3015e5e4624c254a09345475e01c"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.240113 21138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "c988e7c94b20406e9783b933f740c88e"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "60989374ca7be3262bbed005a231302a"
server_key_iv: "856a3015e5e4624c254a09345475e01c"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.241369 21138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:54.242060 21153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.242234 21138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:54.242332 21138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "c988e7c94b20406e9783b933f740c88e"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "60989374ca7be3262bbed005a231302a"
server_key_iv: "856a3015e5e4624c254a09345475e01c"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.242411 21138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:54.260735 21138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:54.263134 21138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:54.263290 21138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:54.267033 21138 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43809
I20260926 22:44:54.267098 21205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43809 every 8 connection(s)
I20260926 22:44:54.267441 21138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:54.268414 21206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:54.270591 21206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Bootstrap starting.
I20260926 22:44:54.270906 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21138
I20260926 22:44:54.271000 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:54.271212 15075 external_mini_cluster.cc:1428] Setting key 4ab2b95ee051c90c0194fa2f881b1a00
I20260926 22:44:54.271301 21206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:54.271621 21206 log.cc:824] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:54.272367 21206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: No bootstrap required, opened a new log
I20260926 22:44:54.274646 21206 raft_consensus.cc:348] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } }
I20260926 22:44:54.274845 21206 raft_consensus.cc:374] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:54.274883 21206 raft_consensus.cc:729] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c988e7c94b20406e9783b933f740c88e, State: Initialized, Role: FOLLOWER
I20260926 22:44:54.275003 21206 consensus_queue.cc:260] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } }
I20260926 22:44:54.275081 21206 raft_consensus.cc:388] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:54.275107 21206 raft_consensus.cc:482] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:54.275149 21206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:54.275714 21206 raft_consensus.cc:504] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } }
I20260926 22:44:54.275846 21206 leader_election.cc:302] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c988e7c94b20406e9783b933f740c88e; no voters:
I20260926 22:44:54.276015 21206 leader_election.cc:288] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:54.276039 21211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:54.276190 21211 raft_consensus.cc:686] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 LEADER]: Becoming Leader. State: Replica: c988e7c94b20406e9783b933f740c88e, State: Running, Role: LEADER
I20260926 22:44:54.276274 21211 consensus_queue.cc:237] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } }
I20260926 22:44:54.276296 21206 sys_catalog.cc:564] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:54.276607 21213 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c988e7c94b20406e9783b933f740c88e. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } }
I20260926 22:44:54.276707 21213 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:54.276937 21216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:54.277420 21216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:54.277791 21212 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } }
I20260926 22:44:54.279053 21212 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:54.280376 21216 catalog_manager.cc:1347] Generated new cluster ID: 3494f7c568e34c0398ab343870a5edf0
I20260926 22:44:54.280441 21216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:54.282258 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.271705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.14.184.254:43809)
0926 22:44:54.271920 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.271929 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.271964 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:44:54.272431 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.272438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.272521 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.272650 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.273438 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.274245 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.275331 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.275722 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.275821 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.278759 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.278763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.278764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.278774 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.280755 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.281118 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.281121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.281121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.281216 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.281432 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.281433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.281434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.281657 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.281815 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.281911 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.282046 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260926 22:44:54.317653 21216 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:54.318210 21216 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:54.322703 21216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Generated new TSK 0
I20260926 22:44:54.322911 21216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab.
Entry for principal oryx/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab.
I20260926 22:44:54.357733 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43809
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:54.453116 21232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:54.453311 21232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:54.453336 21232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:54.454386 21232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:54.454419 21232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:54.454444 21232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:54.455843 21232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:54.456018 21232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:54.456179 21232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:54.461962 21238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.462057 21237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.462172 21240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:54.462438 21232 server_base.cc:1034] running on GCE node
I20260926 22:44:54.462692 21232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:54.462961 21232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:54.464134 21232 hybrid_clock.cc:648] HybridClock initialized: now 1790462694464121 us; error 26 us; skew 500 ppm
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:54.466791 21232 init.cc:377] Logged in from keytab as oryx/127.14.184.193@KRBTEST.COM (short username oryx)
I20260926 22:44:54.467199 21232 webserver.cc:460] Webserver started at http://127.14.184.193:32795/ using document root <none> and password file <none>
I20260926 22:44:54.467442 21232 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:54.467507 21232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:54.467640 21232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:54.468627 21232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "1d02d950a5e04659a5cba4233efd171f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "2e2bfc4deb249196458f595f9dab7769"
server_key_iv: "8eaa1d89a59120cd17eb437596d823c5"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.468988 21232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "1d02d950a5e04659a5cba4233efd171f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "2e2bfc4deb249196458f595f9dab7769"
server_key_iv: "8eaa1d89a59120cd17eb437596d823c5"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.470242 21232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:44:54.470994 21247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.471155 21232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:54.471254 21232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "1d02d950a5e04659a5cba4233efd171f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "2e2bfc4deb249196458f595f9dab7769"
server_key_iv: "8eaa1d89a59120cd17eb437596d823c5"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.471367 21232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:54.498215 21232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:54.500638 21232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:54.500814 21232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:54.501055 21232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:54.501375 21232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:54.501428 21232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.501464 21232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:54.501492 21232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.507193 21232 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:41781
I20260926 22:44:54.507242 21360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:41781 every 8 connection(s)
I20260926 22:44:54.507544 21232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:54.513036 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21232
I20260926 22:44:54.513164 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:54.513402 15075 external_mini_cluster.cc:1428] Setting key 0401d667c10ebbbc6fa57375b7815d43
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:54.517699 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.508193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:40063)
0926 22:44:54.508378 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.508422 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.508966 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.509242 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.509246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.509793 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.509983 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.509988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.510640 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.510640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.511486 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.511488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.511908 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.511909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.512028 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.512542 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:54.512550 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:54.514235 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:54.516271 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.516279 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.516282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.516611 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.516796 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.516797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.516797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.516932 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.517224 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:54.517228 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:54.517339 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 22:44:54.517422 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 22:44:54.517539 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260926 22:44:54.517701 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.508254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40063 (local address 127.14.184.254:43809)
0926 22:44:54.508335 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.508337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.508536 (+ 199us) server_negotiation.cc:407] Connection header received
0926 22:44:54.509092 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.509093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.509104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.509165 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.510079 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.510564 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.511577 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.511823 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.511891 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.514332 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.514333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.514334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.514341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.516186 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.516687 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.516688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.516688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.516738 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.517006 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.517007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.517007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.517158 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.517229 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.517469 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.517564 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 22:44:54.518209 21361 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43809
I20260926 22:44:54.518322 21361 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:54.518507 21361 heartbeater.cc:511] Master 127.14.184.254:43809 requested a full tablet report, sending...
I20260926 22:44:54.519119 21170 ts_manager.cc:194] Registered new tserver with Master: 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781)
I20260926 22:44:54.520138 21170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:40063
WARNING: no policy specified for oryx/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab.
Entry for principal oryx/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab.
I20260926 22:44:54.539529 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43809
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:54.620126 21366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:54.620327 21366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:54.620352 21366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:54.621531 21366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:54.621574 21366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:54.621626 21366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:54.623096 21366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:54.623343 21366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:54.623584 21366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:54.628500 21366 server_base.cc:1034] running on GCE node
W20260926 22:44:54.628473 21374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.628473 21372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.628471 21371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:54.628862 21366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:54.629113 21366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:54.630254 21366 hybrid_clock.cc:648] HybridClock initialized: now 1790462694630239 us; error 30 us; skew 500 ppm
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:54.632647 21366 init.cc:377] Logged in from keytab as oryx/127.14.184.194@KRBTEST.COM (short username oryx)
I20260926 22:44:54.633054 21366 webserver.cc:460] Webserver started at http://127.14.184.194:44115/ using document root <none> and password file <none>
I20260926 22:44:54.633279 21366 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:54.633345 21366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:54.633499 21366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:54.634471 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "106191e21215457892ec751f5e401f76"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "eede75d405a6bde037af377c45c726cd"
server_key_iv: "842745e52c7d7a9746f8f36d85085291"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.634858 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "106191e21215457892ec751f5e401f76"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "eede75d405a6bde037af377c45c726cd"
server_key_iv: "842745e52c7d7a9746f8f36d85085291"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.636122 21366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:54.636812 21381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.636981 21366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:54.637077 21366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "106191e21215457892ec751f5e401f76"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "eede75d405a6bde037af377c45c726cd"
server_key_iv: "842745e52c7d7a9746f8f36d85085291"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.637158 21366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:54.649616 21366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:54.651397 21366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:54.651556 21366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:54.651777 21366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:54.652073 21366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:54.652125 21366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.652155 21366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:54.652184 21366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.657969 21366 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:39213
I20260926 22:44:54.658026 21494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:39213 every 8 connection(s)
I20260926 22:44:54.658339 21366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:54.663950 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21366
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:54.664055 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:54.664233 15075 external_mini_cluster.cc:1428] Setting key c4f45ffe2f8c97ca1d851d566fed0ce7
I20260926 22:44:54.667303 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.659058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:49433 (local address 127.14.184.254:43809)
0926 22:44:54.659140 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.659143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.659338 (+ 195us) server_negotiation.cc:407] Connection header received
0926 22:44:54.659818 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.659819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.659829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.659880 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.660744 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.661219 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.662128 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.662312 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.662361 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.664526 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.664527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.664528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.664537 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.665892 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.666318 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.666320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.666320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.666386 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.666647 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.666648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.666648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.666787 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.666863 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.667133 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.667215 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 22:44:54.667478 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.659020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:49433)
0926 22:44:54.659195 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.659229 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.659734 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.659935 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.659938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.660487 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.660672 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.660675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.661292 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.661293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.662055 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.662057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.662388 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.662389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.662486 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.662947 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:54.662955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:54.664430 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:54.666000 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.666007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.666009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.666227 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.666460 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.666461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.666461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.666577 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.666854 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:54.666861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:54.667003 (+ 142us) client_negotiation.cc:769] Sending connection context
0926 22:44:54.667110 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 22:44:54.667261 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260926 22:44:54.668087 21495 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43809
I20260926 22:44:54.668190 21495 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:54.668365 21495 heartbeater.cc:511] Master 127.14.184.254:43809 requested a full tablet report, sending...
I20260926 22:44:54.668780 21170 ts_manager.cc:194] Registered new tserver with Master: 106191e21215457892ec751f5e401f76 (127.14.184.194:39213)
I20260926 22:44:54.669147 21170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:49433
WARNING: no policy specified for oryx/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab.
Entry for principal oryx/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab.
I20260926 22:44:54.687588 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43809
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:54.768000 21500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:54.768254 21500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:54.768296 21500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:54.769449 21500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:54.769519 21500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:54.769565 21500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:54.771071 21500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:54.771346 21500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:54.771587 21500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:54.776734 21500 server_base.cc:1034] running on GCE node
W20260926 22:44:54.776811 21506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.776705 21508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.776701 21505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:54.777179 21500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:54.777437 21500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:54.778596 21500 hybrid_clock.cc:648] HybridClock initialized: now 1790462694778576 us; error 35 us; skew 500 ppm
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:54.781086 21500 init.cc:377] Logged in from keytab as oryx/127.14.184.195@KRBTEST.COM (short username oryx)
I20260926 22:44:54.781592 21500 webserver.cc:460] Webserver started at http://127.14.184.195:34019/ using document root <none> and password file <none>
I20260926 22:44:54.781829 21500 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:54.781898 21500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:54.782054 21500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:54.783177 21500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "a5d19e92a1a4434b80386a7678cc060f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "388502579a735ff09da4df343cecfe10"
server_key_iv: "bc5b8dbc83a0a3b92d2e3a590d4756e3"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.783550 21500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "a5d19e92a1a4434b80386a7678cc060f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "388502579a735ff09da4df343cecfe10"
server_key_iv: "bc5b8dbc83a0a3b92d2e3a590d4756e3"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.784826 21500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:54.785514 21515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.785674 21500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:54.785779 21500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "a5d19e92a1a4434b80386a7678cc060f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "388502579a735ff09da4df343cecfe10"
server_key_iv: "bc5b8dbc83a0a3b92d2e3a590d4756e3"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.785852 21500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:54.800839 21500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:54.802802 21500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:54.802965 21500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:54.803182 21500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:54.803495 21500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:54.803550 21500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.803591 21500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:54.803617 21500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.809506 21500 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:45289
I20260926 22:44:54.809556 21628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:45289 every 8 connection(s)
I20260926 22:44:54.809860 21500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:54.811666 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21500
I20260926 22:44:54.811743 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:54.811928 15075 external_mini_cluster.cc:1428] Setting key 12af287db05975dab78ef51e16c6d43a
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:54.818635 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.810500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:38745 (local address 127.14.184.254:43809)
0926 22:44:54.810564 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.810565 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.810796 (+ 231us) server_negotiation.cc:407] Connection header received
0926 22:44:54.811257 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.811259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.811269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.811334 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.812475 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.813030 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.813977 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.814159 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.814225 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.816363 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.816364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.816365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.816373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.817541 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.817886 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.817887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.817887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.817936 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.818137 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.818137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.818137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.818239 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.818288 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.818511 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.818572 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 22:44:54.818836 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.810453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:38745)
0926 22:44:54.810605 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.810647 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.811166 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.811387 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.811391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.812102 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.812380 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.812387 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.813107 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.813108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.813891 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.813893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.814250 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.814251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.814356 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.814969 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:54.814978 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:54.816283 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:54.817611 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.817616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.817618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.817823 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.817993 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.817994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.817994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.818083 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.818292 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:54.818295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:54.818401 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 22:44:54.818480 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 22:44:54.818590 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260926 22:44:54.819258 21629 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43809
I20260926 22:44:54.819334 21629 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:54.819473 21629 heartbeater.cc:511] Master 127.14.184.254:43809 requested a full tablet report, sending...
I20260926 22:44:54.819774 21170 ts_manager.cc:194] Registered new tserver with Master: a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289)
I20260926 22:44:54.820130 21170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:38745
I20260926 22:44:54.823789 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.184.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.253.keytab.
Entry for principal oryx/127.14.184.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.253.keytab.
I20260926 22:44:54.847246 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:34531
--webserver_interface=127.14.184.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.253.keytab
--principal=oryx/127.14.184.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43809,127.14.184.253:34531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:54.926251 21634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:54.926440 21634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:54.926461 21634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:54.927628 21634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:54.927668 21634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:54.927682 21634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:54.927695 21634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:54.927729 21634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:54.929158 21634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43809,127.14.184.253:34531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.253.keytab
--principal=oryx/127.14.184.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:34531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.184.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
W20260926 22:44:54.929250 21634 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.184.254:43809,127.14.184.253:34531'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 22:44:54.929426 21634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:54.929651 21634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:54.934202 21640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.934350 21642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:54.934206 21639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:54.934240 21634 server_base.cc:1034] running on GCE node
I20260926 22:44:54.934841 21634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:54.935194 21634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:54.936362 21634 hybrid_clock.cc:648] HybridClock initialized: now 1790462694936351 us; error 27 us; skew 500 ppm
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:54.939519 21634 init.cc:377] Logged in from keytab as oryx/127.14.184.253@KRBTEST.COM (short username oryx)
I20260926 22:44:54.940052 21634 webserver.cc:460] Webserver started at http://127.14.184.253:43911/ using document root <none> and password file <none>
I20260926 22:44:54.940295 21634 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:54.940347 21634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:54.940474 21634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:54.942001 21634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/instance:
uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "6c15f7a606ca5f61325e83dfe54f60e0"
server_key_iv: "679347c0722908ce4c0b9d12559fc96e"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.942407 21634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal/instance:
uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "6c15f7a606ca5f61325e83dfe54f60e0"
server_key_iv: "679347c0722908ce4c0b9d12559fc96e"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.944130 21634 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 22:44:54.945051 21649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:54.945209 21634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:54.945294 21634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "6c15f7a606ca5f61325e83dfe54f60e0"
server_key_iv: "679347c0722908ce4c0b9d12559fc96e"
server_key_version: "encryptionkey@0"
I20260926 22:44:54.945361 21634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:54.962798 21634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:54.964475 21634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:54.964581 21634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:54.968202 21634 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.253:34531
I20260926 22:44:54.968250 21701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.253:34531 every 8 connection(s)
I20260926 22:44:54.968568 21634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
I20260926 22:44:54.969504 21702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:54.971113 21702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:44:54.971601 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21634
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.253@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
I20260926 22:44:54.980321 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.971882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.14.184.254:43809)
0926 22:44:54.971946 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.971949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.972163 (+ 214us) server_negotiation.cc:407] Connection header received
0926 22:44:54.972645 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.972647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.972660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.972712 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.973245 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.973845 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.974793 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.975086 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.975160 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.977282 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.977283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.977283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.977290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.978814 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.979285 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.979287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.979287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.979355 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.979631 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.979632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.979632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.979784 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.979838 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.980190 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.980258 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:44:54.980635 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.971849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:35758)
0926 22:44:54.972030 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.972069 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.972551 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.972728 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.972730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.973002 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.973146 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.973150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.973937 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.973938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.974627 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.974629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.975174 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.975175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.975301 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.975816 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:54.975824 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:54.977184 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:54.978891 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.978903 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.978906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.979193 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.979418 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.979419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.979419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.979553 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.979855 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:54.979860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:54.980023 (+ 163us) client_negotiation.cc:769] Sending connection context
0926 22:44:54.980140 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 22:44:54.980303 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260926 22:44:54.981189 21702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:44:54.981479 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.972105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.14.184.253:34531)
0926 22:44:54.972240 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.972245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.972323 (+ 78us) server_negotiation.cc:407] Connection header received
0926 22:44:54.972566 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.972576 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.972597 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.972685 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.973403 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.974240 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.975162 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.975582 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.975634 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.978288 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.978294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.978294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.978301 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.980098 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.980506 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.980509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.980509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.980570 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.980800 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.980802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.980802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.980957 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.981097 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.981305 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.981378 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
Sep 26 22:44:54 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.253@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
I20260926 22:44:54.987725 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.981445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:37828)
0926 22:44:54.981580 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.981590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.981633 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.981757 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.981758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.981894 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.981932 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.981933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.982570 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.982571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.983098 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.983100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.983409 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.983410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.983459 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.983849 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:54.983853 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:54.984889 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:54.986940 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.986941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.986942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.987091 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.987279 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:54.987279 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:54.987279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:54.987324 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:54.987520 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:54.987520 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:54.987551 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 22:44:54.987600 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 22:44:54.987669 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260926 22:44:54.987902 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.981476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.14.184.253:34531)
0926 22:44:54.981595 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.981596 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.981619 (+ 23us) server_negotiation.cc:407] Connection header received
0926 22:44:54.981686 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.981687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.981696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.981756 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.982028 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.982504 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.983162 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.983350 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.983401 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.984982 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:54.984983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:54.984983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:54.984989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:54.986863 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.987164 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.987165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.987166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.987217 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:54.987376 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:54.987377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:54.987377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:54.987468 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:54.987518 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:54.987630 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 22:44:54.987715 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260926 22:44:54.989241 21702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Bootstrap starting.
I20260926 22:44:54.989836 21702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:54.990062 21702 log.cc:824] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:54.990739 21702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: No bootstrap required, opened a new log
I20260926 22:44:54.992076 21702 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } }
I20260926 22:44:54.992193 21702 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:54.992215 21702 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f853b10e6a4ec4b69d8bec2a7d927d, State: Initialized, Role: FOLLOWER
I20260926 22:44:54.992301 21702 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } }
I20260926 22:44:54.992555 21702 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:54.992523 21709 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } } }
I20260926 22:44:54.992623 21709 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: FOLLOWER
W20260926 22:44:54.994271 21720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:44:54.994340 21720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:44:54.995863 21634 master_runner.cc:428] Detected that this master f6f853b10e6a4ec4b69d8bec2a7d927d is joining an existing cluster
I20260926 22:44:54.995921 21634 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.184.253:34531
I20260926 22:44:55.003001 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.997138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.14.184.253:34531)
0926 22:44:54.997173 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.997175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.997247 (+ 72us) server_negotiation.cc:407] Connection header received
0926 22:44:54.997318 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.997318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.997327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.997352 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.997703 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.998175 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.998953 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.999171 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.999252 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.000409 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.000411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.000411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.000417 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.001836 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.002170 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.002171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.002172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.002221 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.002401 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.002402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.002402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.002508 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.002579 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.002756 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.002874 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:55.003297 21727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.997096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:37830)
0926 22:44:54.997197 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.997205 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.997259 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.997398 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.997398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.997588 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.997634 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.997635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.998247 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.998248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.998859 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.998861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.999281 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.999282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.999355 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:54.999780 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:54.999785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.000111 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.001919 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.001919 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.001920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.002083 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.002289 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.002290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.002290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.002342 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.002570 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.002571 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.002608 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.002732 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.002964 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":109,"thread_start_us":54,"threads_started":1}
I20260926 22:44:55.004009 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.996930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.14.184.254:43809)
0926 22:44:54.996978 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:54.996980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:54.997119 (+ 139us) server_negotiation.cc:407] Connection header received
0926 22:44:54.997233 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:54.997235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:54.997247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:54.997304 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:54.997746 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.998508 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.999598 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.999910 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.999952 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.000904 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.000905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.000905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.000912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.002614 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.003120 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.003121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.003121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.003176 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.003403 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.003404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.003405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.003541 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.003610 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.003878 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.003949 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:44:55.004057 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:54.996851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:35766)
0926 22:44:54.997050 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:54.997062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:54.997147 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:54.997313 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:54.997313 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:54.997561 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:54.997648 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.997649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:54.998610 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:54.998611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:54.999495 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:54.999498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.000031 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.000032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.000094 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.000424 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.000429 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.000790 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.002738 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.002739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.002739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.003030 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.003245 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.003246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.003246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.003320 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.003674 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.003675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.003744 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.003841 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.003956 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.254@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
I20260926 22:44:55.012668 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.006272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.14.184.253:34531)
0926 22:44:55.006306 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.006307 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.006314 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:55.006388 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.006389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.006408 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.006456 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.006902 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.007327 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.008077 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.008266 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.008335 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.010191 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.010192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.010192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.010199 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.011493 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.011882 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.011883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.011884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.011942 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.012136 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.012137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.012137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.012237 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.012309 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.012492 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.012574 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:55.012701 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.006089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:37846)
0926 22:44:55.006210 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.006253 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.006320 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.006459 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.006463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.006773 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.006820 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.006822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.007398 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.007399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.007982 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.007983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.008336 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.008337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.008416 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.008844 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.008848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.010107 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.011585 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.011592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.011594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.011790 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.012006 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.012007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.012007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.012070 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.012300 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.012306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.012368 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.012474 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.012589 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260926 22:44:55.013283 21170 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.184.253:34531: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
member_type: NON_VOTER
last_known_addr {
host: "127.14.184.253"
port: 34531
}
attrs {
promote: true
}
}
dest_uuid: "c988e7c94b20406e9783b933f740c88e"
cas_config_opid_index: -1
I20260926 22:44:55.013619 21170 consensus_queue.cc:237] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } }
I20260926 22:44:55.014123 21730 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } }
I20260926 22:44:55.014449 21676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:55.015257 21730 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.184.253:34531
I20260926 22:44:55.015266 21676 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Refusing update from remote peer c988e7c94b20406e9783b933f740c88e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 22:44:55.015476 21634 master.cc:537] Master@127.14.184.253:34531 shutting down...
I20260926 22:44:55.015646 21731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:55.015743 21729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } }
I20260926 22:44:55.015952 21729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
W20260926 22:44:55.016402 21156 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 22:44:55.016785 21156 consensus_peers.cc:487] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e -> Peer f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253:34531): Couldn't send request to peer f6f853b10e6a4ec4b69d8bec2a7d927d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 22:44:55.018204 21634 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:55.018311 21634 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:55.018362 21634 tablet_replica.cc:330] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: stopping tablet replica
I20260926 22:44:55.031307 21634 master.cc:552] Master@127.14.184.253:34531 shutdown complete.
I20260926 22:44:55.031365 21634 master_runner.cc:305] Clearing existing system tablet
I20260926 22:44:55.031497 21634 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.031545 21634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.032275 21634 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:55.032974 21735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.033144 21634 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.033211 21634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.033243 21634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "6c15f7a606ca5f61325e83dfe54f60e0"
server_key_iv: "679347c0722908ce4c0b9d12559fc96e"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.033942 21634 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:55.035347 21634 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 22:44:55.035434 21634 log.cc:1196] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 22:44:55.035633 21634 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Deleting consensus metadata
I20260926 22:44:55.035776 21634 master_runner.cc:315] Copying system tablet from 127.14.184.254:43809
I20260926 22:44:55.036407 21634 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: tablet copy: Beginning tablet copy session from remote peer at address 127.14.184.254:43809
I20260926 22:44:55.042846 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.036782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.14.184.254:43809)
0926 22:44:55.036826 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.036828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.036878 (+ 50us) server_negotiation.cc:407] Connection header received
0926 22:44:55.036959 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.036960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.036968 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.037011 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.037371 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.037855 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.038979 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.039197 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.039262 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.040285 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.040286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.040286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.040292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.041714 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.042106 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.042107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.042107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.042156 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.042347 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.042347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.042348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.042443 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.042492 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.042653 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.042784 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:55.042874 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.036680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:35782)
0926 22:44:55.036802 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.036814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.036896 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.037027 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.037027 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.037232 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.037299 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.037301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.037931 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.037932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.038899 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.038901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.039275 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.039276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.039346 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.039771 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.039776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.040180 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.041793 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.041794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.041795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.042029 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.042217 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.042218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.042218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.042285 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.042506 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.042507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.042555 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.042630 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.042753 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260926 22:44:55.043172 21190 tablet_copy_service.cc:140] P c988e7c94b20406e9783b933f740c88e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f6f853b10e6a4ec4b69d8bec2a7d927d ({username='oryx', principal='oryx/127.14.184.253@KRBTEST.COM'} at 127.0.0.1:35782)
I20260926 22:44:55.043282 21190 tablet_copy_service.cc:161] P c988e7c94b20406e9783b933f740c88e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f6f853b10e6a4ec4b69d8bec2a7d927d at {username='oryx', principal='oryx/127.14.184.253@KRBTEST.COM'} at 127.0.0.1:35782: session id = f6f853b10e6a4ec4b69d8bec2a7d927d-00000000000000000000000000000000
I20260926 22:44:55.044142 21190 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 22:44:55.044798 21634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 22:44:55.045766 21634 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: tablet copy: Starting download of 0 data blocks...
I20260926 22:44:55.045969 21634 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: tablet copy: Starting download of 1 WAL segments...
I20260926 22:44:55.046868 21634 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 22:44:55.047631 21190 tablet_copy_service.cc:342] P c988e7c94b20406e9783b933f740c88e: Request end of tablet copy session f6f853b10e6a4ec4b69d8bec2a7d927d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.184.253@KRBTEST.COM'} at 127.0.0.1:35782
I20260926 22:44:55.047698 21190 tablet_copy_service.cc:433] P c988e7c94b20406e9783b933f740c88e: ending tablet copy session f6f853b10e6a4ec4b69d8bec2a7d927d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f6f853b10e6a4ec4b69d8bec2a7d927d
I20260926 22:44:55.050011 21634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:55.051296 21742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.051335 21743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.051345 21745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.055178 21634 server_base.cc:1034] running on GCE node
I20260926 22:44:55.055282 21634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:55.055450 21634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:55.056577 21634 hybrid_clock.cc:648] HybridClock initialized: now 1790462695056570 us; error 24 us; skew 500 ppm
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:55.057843 21634 init.cc:377] Logged in from keytab as oryx/127.14.184.253@KRBTEST.COM (short username oryx)
I20260926 22:44:55.058116 21634 webserver.cc:460] Webserver started at http://127.14.184.253:35899/ using document root <none> and password file <none>
I20260926 22:44:55.058249 21634 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.058290 21634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.059043 21634 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:55.059568 21752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.059685 21634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:55.059731 21634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "6c15f7a606ca5f61325e83dfe54f60e0"
server_key_iv: "679347c0722908ce4c0b9d12559fc96e"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.059785 21634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.080880 21634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:55.082427 21634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:55.085498 21634 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.253:34531
I20260926 22:44:55.085515 21804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.253:34531 every 8 connection(s)
I20260926 22:44:55.085822 21634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
I20260926 22:44:55.086709 21805 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:44:55.087256 21805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Bootstrap starting.
I20260926 22:44:55.090130 21805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:44:55.090374 21805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: Bootstrap complete.
I20260926 22:44:55.090571 21805 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } }
I20260926 22:44:55.090660 21805 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f6f853b10e6a4ec4b69d8bec2a7d927d, State: Initialized, Role: LEARNER
I20260926 22:44:55.090726 21805 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } }
I20260926 22:44:55.090926 21805 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:55.090903 21808 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } }
I20260926 22:44:55.091032 21808 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: LEARNER
I20260926 22:44:55.092510 21819 catalog_manager.cc:1259] Loaded cluster ID: 3494f7c568e34c0398ab343870a5edf0
I20260926 22:44:55.092552 21819 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: loading cluster ID for follower catalog manager: success
I20260926 22:44:55.093425 21819 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: acquiring CA information for follower catalog manager: success
I20260926 22:44:55.093684 21819 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:44:55.139705 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.133632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.14.184.253:34531)
0926 22:44:55.133797 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.133800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.133808 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:55.133841 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.133841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.133849 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.133944 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.134493 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.135215 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.135933 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.136123 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.136208 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.137335 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.137336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.137337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.137343 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.138505 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.138915 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.138916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.138916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.138975 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.139181 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.139181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.139182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.139296 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.139422 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.139518 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.139597 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for oryx/127.14.184.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.252.keytab.
Entry for principal oryx/127.14.184.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.252.keytab.
I20260926 22:44:55.162353 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:40827
--webserver_interface=127.14.184.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.252.keytab
--principal=oryx/127.14.184.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:55.241195 21823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:55.241412 21823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:55.241456 21823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:55.242576 21823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:55.242645 21823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:55.242668 21823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:55.242709 21823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:55.242754 21823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:55.244187 21823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.252.keytab
--principal=oryx/127.14.184.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:40827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.184.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:55.244452 21823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:55.244707 21823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:55.249384 21831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.249374 21828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.249560 21823 server_base.cc:1034] running on GCE node
W20260926 22:44:55.249622 21829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.249933 21823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:55.250167 21823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:55.251307 21823 hybrid_clock.cc:648] HybridClock initialized: now 1790462695251295 us; error 28 us; skew 500 ppm
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:55.253621 21823 init.cc:377] Logged in from keytab as oryx/127.14.184.252@KRBTEST.COM (short username oryx)
I20260926 22:44:55.254043 21823 webserver.cc:460] Webserver started at http://127.14.184.252:36481/ using document root <none> and password file <none>
I20260926 22:44:55.254215 21823 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.254253 21823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.254336 21823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:55.255415 21823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/instance:
uuid: "f088109a197045c68e1552445490331b"
format_stamp: "Formatted at 2026-09-26 22:44:55 on dist-test-slave-3986"
server_key: "c11d935d553c2374af54806e799bce29"
server_key_iv: "c11cd8ac98e5931c935902584c184a94"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.255697 21823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal/instance:
uuid: "f088109a197045c68e1552445490331b"
format_stamp: "Formatted at 2026-09-26 22:44:55 on dist-test-slave-3986"
server_key: "c11d935d553c2374af54806e799bce29"
server_key_iv: "c11cd8ac98e5931c935902584c184a94"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.256942 21823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:55.257579 21838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.257754 21823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:55.257839 21823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
uuid: "f088109a197045c68e1552445490331b"
format_stamp: "Formatted at 2026-09-26 22:44:55 on dist-test-slave-3986"
server_key: "c11d935d553c2374af54806e799bce29"
server_key_iv: "c11cd8ac98e5931c935902584c184a94"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.257910 21823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.289242 21823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:55.291164 21823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:55.291275 21823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:55.295413 21823 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.252:40827
I20260926 22:44:55.295444 21890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.252:40827 every 8 connection(s)
I20260926 22:44:55.295789 21823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
I20260926 22:44:55.296943 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21823
I20260926 22:44:55.296976 21891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:55.299208 21891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.184.252@KRBTEST.COM
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.252@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:55.307327 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.297591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.14.184.252:40827)
0926 22:44:55.297783 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.297792 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.297826 (+ 34us) server_negotiation.cc:407] Connection header received
0926 22:44:55.298475 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.298484 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.298549 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.298722 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.299307 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.300502 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.301234 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.301646 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.301722 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.304335 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.304343 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.304343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.304350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.305835 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.306185 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.306188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.306188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.306267 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.306470 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.306471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.306471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.306752 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.306916 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.306990 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.307109 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260926 22:44:55.307559 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.299875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:35798)
0926 22:44:55.300033 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.300071 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.300146 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.300289 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.300291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.300522 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.300624 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.300626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.301287 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.301288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.301993 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.301995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.302440 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.302441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.302518 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.302960 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.302970 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.304790 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.306427 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.306434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.306436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.306635 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.306884 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.306885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.306885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.306949 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.307216 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.307220 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.307264 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.307349 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.307465 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
I20260926 22:44:55.307571 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.299987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.14.184.254:43809)
0926 22:44:55.300037 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.300039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.300135 (+ 96us) server_negotiation.cc:407] Connection header received
0926 22:44:55.300192 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.300193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.300202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.300265 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.300705 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.301168 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.302069 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.302342 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.302399 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.304875 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.304877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.304877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.304883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.306303 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.306758 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.306759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.306760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.306816 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.307014 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.307015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.307015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.307142 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.307208 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.307398 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.307488 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 22:44:55.307966 21891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.252@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
I20260926 22:44:55.313565 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.308194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:37876)
0926 22:44:55.308242 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.308252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.308285 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.308451 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.308452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.308582 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.308607 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.308608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.309246 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.309247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.309732 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.309734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.310105 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.310106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.310159 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.310489 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.310494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.311586 (+ 1092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.312837 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.312838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.312838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.312988 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.313150 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.313151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.313151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.313196 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.313364 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.313365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.313401 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.313450 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.313511 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:44:55.313581 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.308278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.14.184.253:34531)
0926 22:44:55.308326 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.308328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.308336 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:55.308361 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.308362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.308380 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.308449 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.308686 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.309160 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.309814 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.310034 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.310107 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.311665 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.311666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.311666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.311672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.312771 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.313050 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.313051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.313051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.313095 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.313236 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.313237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.313237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.313319 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.313362 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.313465 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.313525 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:55.314023 21891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.252@KRBTEST.COM for oryx/127.14.184.252@KRBTEST.COM
I20260926 22:44:55.319682 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.314257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:39538)
0926 22:44:55.314308 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.314318 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.314356 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.314465 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.314466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.314563 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.314586 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.314587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.315179 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.315180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.315634 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.315635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.315946 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.315947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.315980 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.316278 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.316281 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.317293 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.318782 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.318783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.318783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.318925 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.319093 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.319094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.319094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.319149 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.319387 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.319388 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.319424 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.319491 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.319581 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 22:44:55.319736 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.314310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39538 (local address 127.14.184.252:40827)
0926 22:44:55.314351 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.314352 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.314359 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:55.314416 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.314416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.314423 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.314462 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.314638 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.315116 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.315697 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.315888 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.315935 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.317360 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.317361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.317362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.317368 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.318725 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.318978 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.318978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.318978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.319025 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.319221 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.319222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.319223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.319318 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.319393 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.319516 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.319629 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":19}
I20260926 22:44:55.321048 21891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Bootstrap starting.
I20260926 22:44:55.321866 21891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:55.322161 21891 log.cc:824] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:55.322903 21891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: No bootstrap required, opened a new log
I20260926 22:44:55.324280 21891 raft_consensus.cc:348] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } }
I20260926 22:44:55.324406 21891 raft_consensus.cc:374] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:55.324452 21891 raft_consensus.cc:729] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f088109a197045c68e1552445490331b, State: Initialized, Role: FOLLOWER
I20260926 22:44:55.324581 21891 consensus_queue.cc:260] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } }
I20260926 22:44:55.324834 21896 sys_catalog.cc:455] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } } }
I20260926 22:44:55.324872 21891 sys_catalog.cc:564] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:55.324936 21896 sys_catalog.cc:458] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: This master's current role is: FOLLOWER
W20260926 22:44:55.326509 21907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:44:55.326584 21907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:44:55.328737 21823 master_runner.cc:428] Detected that this master f088109a197045c68e1552445490331b is joining an existing cluster
I20260926 22:44:55.328804 21823 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.184.252:40827
I20260926 22:44:55.333122 21896 raft_consensus.cc:482] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:55.333225 21896 raft_consensus.cc:504] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } }
I20260926 22:44:55.333503 21896 leader_election.cc:288] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809), f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253:34531)
I20260926 22:44:55.333814 21180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f088109a197045c68e1552445490331b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c988e7c94b20406e9783b933f740c88e" is_pre_election: true
I20260926 22:44:55.334131 21779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f088109a197045c68e1552445490331b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" is_pre_election: true
I20260926 22:44:55.334260 21779 raft_consensus.cc:1799] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Handling vote request from an unknown peer f088109a197045c68e1552445490331b
I20260926 22:44:55.334319 21779 raft_consensus.cc:2396] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate f088109a197045c68e1552445490331b for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260926 22:44:55.334515 21841 leader_election.cc:302] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f088109a197045c68e1552445490331b; no voters: c988e7c94b20406e9783b933f740c88e, f6f853b10e6a4ec4b69d8bec2a7d927d
I20260926 22:44:55.334640 21896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:55.335464 21896 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 22:44:55.338299 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.330026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37892 (local address 127.14.184.253:34531)
0926 22:44:55.330067 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.330069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.330174 (+ 105us) server_negotiation.cc:407] Connection header received
0926 22:44:55.330288 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.330290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.330302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.330335 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.330903 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.331670 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.332678 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.333001 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.333043 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.335704 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.335706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.335706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.335714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.337147 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.337510 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.337512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.337512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.337567 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.337787 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.337788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.337789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.337907 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.337981 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.338130 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.338208 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:55.338315 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.329983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:37892)
0926 22:44:55.330114 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.330124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.330211 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.330422 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.330423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.330726 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.330809 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.330811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.331757 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.331758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.332589 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.332591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.333779 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.333780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.333842 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.334199 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.334204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.335601 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.337248 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.337249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.337249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.337426 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.337639 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.337640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.337641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.337716 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.337973 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.337973 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.338017 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.338105 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.338210 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":24,"thread_start_us":59,"threads_started":1}
I20260926 22:44:55.358645 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21232
I20260926 22:44:55.362632 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.329817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.14.184.254:43809)
0926 22:44:55.329890 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.329893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.329986 (+ 93us) server_negotiation.cc:407] Connection header received
0926 22:44:55.330089 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.330090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.330100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.330130 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.330580 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.331384 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.332557 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.332904 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.332950 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.359321 (+ 26371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.359323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.359324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.359332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.361092 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.361809 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.361811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.361811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.361877 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.362072 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.362073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.362073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.362206 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.362246 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.362448 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.362533 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:44:55.363242 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.329738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:35804)
0926 22:44:55.329913 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.329924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.330013 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.330209 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.330210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.330401 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.330481 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.330483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.331491 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.331492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.332450 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.332452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.333116 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.333116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.333183 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.333731 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.333736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.359200 (+ 25464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.361219 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.361220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.361220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.361719 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.361950 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.361950 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.361951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.362006 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.362291 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.362292 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.362338 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.362852 (+ 514us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.363133 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260926 22:44:55.365331 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.330266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.14.184.252:40827)
0926 22:44:55.330298 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.330300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.330385 (+ 85us) server_negotiation.cc:407] Connection header received
0926 22:44:55.330475 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.330476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.330484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.330510 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.330908 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.331614 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.332676 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.332982 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.333033 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.360044 (+ 27011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.360046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.360046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.360052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.362598 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.364009 (+ 1411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.364010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.364010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.364057 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.364386 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.364387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.364387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.364523 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.364571 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.365101 (+ 530us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.365182 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:55.365660 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.330136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:39546)
0926 22:44:55.330338 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.330347 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.330414 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.330559 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.330560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.330768 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.330825 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.330826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.331699 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.331700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.332581 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.332584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.333408 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.333408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.333472 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.359196 (+ 25724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.359203 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.359950 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.362703 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.362704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.362705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.363939 (+ 1234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.364115 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.364116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.364116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.364327 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.364776 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.364777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.364819 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.365405 (+ 586us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.365502 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260926 22:44:55.366465 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:41781
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:43809
--webserver_port=32795
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462694, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.254@KRBTEST.COM for oryx/127.14.184.252@KRBTEST.COM
I20260926 22:44:55.373307 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.365820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:39562)
0926 22:44:55.365922 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.365932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.365969 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.366195 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.366196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.366326 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.366362 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.366363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.367160 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.367162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.367737 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.367739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.368233 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.368234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.368315 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.368708 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.368713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.370296 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.372108 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.372109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.372110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.372350 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.372566 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.372567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.372567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.372638 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.372920 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.372921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.372975 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.373053 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.373169 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 22:44:55.373247 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.366032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.14.184.252:40827)
0926 22:44:55.366064 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.366066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.366072 (+ 6us) server_negotiation.cc:407] Connection header received
0926 22:44:55.366099 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.366100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.366109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.366149 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.366452 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.367082 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.367838 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.368149 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.368217 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.370401 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.370403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.370403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.370409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.372018 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.372432 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.372433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.372434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.372491 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.372708 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.372709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.372709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.372828 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.372897 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.373090 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.373163 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:44:55.374214 21170 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.184.252:40827: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f088109a197045c68e1552445490331b"
member_type: NON_VOTER
last_known_addr {
host: "127.14.184.252"
port: 40827
}
attrs {
promote: true
}
}
dest_uuid: "c988e7c94b20406e9783b933f740c88e"
cas_config_opid_index: 5
I20260926 22:44:55.374449 21170 consensus_queue.cc:237] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } }
I20260926 22:44:55.374831 21729 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f088109a197045c68e1552445490331b (127.14.184.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.376061 21729 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.184.252:40827
I20260926 22:44:55.376144 21729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.376232 21729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.376456 21823 master.cc:537] Master@127.14.184.252:40827 shutting down...
I20260926 22:44:55.376665 21865 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Refusing update from remote peer c988e7c94b20406e9783b933f740c88e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 22:44:55.377112 21731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 22:44:55.377758 21156 consensus_peers.cc:487] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e -> Peer f088109a197045c68e1552445490331b (127.14.184.252:40827): Couldn't send request to peer f088109a197045c68e1552445490331b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 22:44:55.380075 21823 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:55.380156 21823 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:55.380196 21823 tablet_replica.cc:330] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: stopping tablet replica
I20260926 22:44:55.381698 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.374890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:37894)
0926 22:44:55.377861 (+ 2971us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.377870 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.377920 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.378143 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.378144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.378145 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.378181 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.378182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.379120 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.379120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.380592 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.380595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.381465 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.381466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.381530 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.381549 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.381596 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.381632 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2957}
I20260926 22:44:55.381937 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.374987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.14.184.253:34531)
0926 22:44:55.377832 (+ 2845us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.377835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.377985 (+ 150us) server_negotiation.cc:407] Connection header received
0926 22:44:55.378010 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.378011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.378034 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.378071 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.378243 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.379031 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.380692 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.381300 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.381365 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.381432 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.381823 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.381862 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2818}
I20260926 22:44:55.382181 21779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Refusing update from remote peer c988e7c94b20406e9783b933f740c88e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 22:44:55.382412 21731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 22:44:55.383138 21779 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f088109a197045c68e1552445490331b (127.14.184.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.384428 21731 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: attempting to promote NON_VOTER f6f853b10e6a4ec4b69d8bec2a7d927d to VOTER
I20260926 22:44:55.384454 21808 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c988e7c94b20406e9783b933f740c88e. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.384536 21808 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: LEARNER
I20260926 22:44:55.384572 21808 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: NON_VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: true } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.384611 21808 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: LEARNER
I20260926 22:44:55.384637 21731 consensus_queue.cc:237] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } }
I20260926 22:44:55.385102 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.384971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:39566)
0926 22:44:55.385029 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.385076 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:40827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 22:44:55.385110 21779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 LEARNER]: Refusing update from remote peer c988e7c94b20406e9783b933f740c88e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 22:44:55.385288 21729 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 22:44:55.385542 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.385466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:39572)
0926 22:44:55.385495 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.385522 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:40827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 22:44:55.385720 21156 consensus_peers.cc:487] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e -> Peer f088109a197045c68e1552445490331b (127.14.184.252:40827): Couldn't send request to peer f088109a197045c68e1552445490331b. Status: Network error: Client connection negotiation failed: client connection to 127.14.184.252:40827: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 22:44:55.385972 21730 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.386113 21779 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.386935 21730 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.387012 21730 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.386978 21729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.387055 21729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.387246 21918 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.387334 21918 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:44:55.395784 21823 master.cc:552] Master@127.14.184.252:40827 shutdown complete.
I20260926 22:44:55.395836 21823 master_runner.cc:305] Clearing existing system tablet
I20260926 22:44:55.395978 21823 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.396026 21823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.396737 21823 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:55.397257 21920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.397403 21823 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.397454 21823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.397475 21823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
uuid: "f088109a197045c68e1552445490331b"
format_stamp: "Formatted at 2026-09-26 22:44:55 on dist-test-slave-3986"
server_key: "c11d935d553c2374af54806e799bce29"
server_key_iv: "c11cd8ac98e5931c935902584c184a94"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.398130 21823 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:55.399572 21823 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 22:44:55.399641 21823 log.cc:1196] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 22:44:55.399817 21823 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Deleting consensus metadata
I20260926 22:44:55.399942 21823 master_runner.cc:315] Copying system tablet from 127.14.184.254:43809
I20260926 22:44:55.400516 21823 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: tablet copy: Beginning tablet copy session from remote peer at address 127.14.184.254:43809
I20260926 22:44:55.407567 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.400859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.14.184.254:43809)
0926 22:44:55.400916 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.400920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.400991 (+ 71us) server_negotiation.cc:407] Connection header received
0926 22:44:55.401063 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.401064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.401075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.401143 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.401487 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.402243 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.403336 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.403615 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.403684 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.404747 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.404749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.404749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.404756 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.406353 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.406787 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.406788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.406789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.406848 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.407061 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.407062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.407062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.407185 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.407250 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.407410 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.407484 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 22:44:55.407568 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.400794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:35814)
0926 22:44:55.400928 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.400939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.401007 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.401147 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.401147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.401351 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.401402 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.401405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.402336 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.402337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.403247 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.403250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.403692 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.403693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.403771 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.404274 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.404280 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.404657 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.406443 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.406444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.406445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.406711 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.406925 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.406926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.406926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.406994 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.407254 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.407255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.407300 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.407374 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.407474 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260926 22:44:55.408162 21190 tablet_copy_service.cc:140] P c988e7c94b20406e9783b933f740c88e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f088109a197045c68e1552445490331b ({username='oryx', principal='oryx/127.14.184.252@KRBTEST.COM'} at 127.0.0.1:35814)
I20260926 22:44:55.408224 21190 tablet_copy_service.cc:161] P c988e7c94b20406e9783b933f740c88e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f088109a197045c68e1552445490331b at {username='oryx', principal='oryx/127.14.184.252@KRBTEST.COM'} at 127.0.0.1:35814: session id = f088109a197045c68e1552445490331b-00000000000000000000000000000000
I20260926 22:44:55.408792 21190 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 22:44:55.409233 21823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 22:44:55.410316 21823 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: tablet copy: Starting download of 0 data blocks...
I20260926 22:44:55.410449 21823 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: tablet copy: Starting download of 1 WAL segments...
I20260926 22:44:55.411204 21823 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 22:44:55.412035 21190 tablet_copy_service.cc:342] P c988e7c94b20406e9783b933f740c88e: Request end of tablet copy session f088109a197045c68e1552445490331b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.184.252@KRBTEST.COM'} at 127.0.0.1:35814
I20260926 22:44:55.412082 21190 tablet_copy_service.cc:433] P c988e7c94b20406e9783b933f740c88e: ending tablet copy session f088109a197045c68e1552445490331b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f088109a197045c68e1552445490331b
I20260926 22:44:55.415118 21823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:55.416380 21927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.416500 21930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.416638 21823 server_base.cc:1034] running on GCE node
W20260926 22:44:55.418819 21928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.419027 21823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:55.419165 21823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:55.420293 21823 hybrid_clock.cc:648] HybridClock initialized: now 1790462695420269 us; error 39 us; skew 500 ppm
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:55.421782 21823 init.cc:377] Logged in from keytab as oryx/127.14.184.252@KRBTEST.COM (short username oryx)
I20260926 22:44:55.422046 21823 webserver.cc:460] Webserver started at http://127.14.184.252:46655/ using document root <none> and password file <none>
I20260926 22:44:55.422166 21823 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.422206 21823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.422955 21823 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:55.423535 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.423707 21823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:55.423758 21823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
uuid: "f088109a197045c68e1552445490331b"
format_stamp: "Formatted at 2026-09-26 22:44:55 on dist-test-slave-3986"
server_key: "c11d935d553c2374af54806e799bce29"
server_key_iv: "c11cd8ac98e5931c935902584c184a94"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.423813 21823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.448477 21823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:55.451122 21823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:55.454315 21823 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.252:40827
I20260926 22:44:55.454416 21989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.252:40827 every 8 connection(s)
I20260926 22:44:55.454624 21823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
I20260926 22:44:55.458621 21990 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:44:55.459172 21990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Bootstrap starting.
I20260926 22:44:55.462138 21990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:44:55.462353 21990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Bootstrap complete.
I20260926 22:44:55.462553 21990 raft_consensus.cc:348] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } }
I20260926 22:44:55.462641 21990 raft_consensus.cc:729] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f088109a197045c68e1552445490331b, State: Initialized, Role: LEARNER
I20260926 22:44:55.462702 21990 consensus_queue.cc:260] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } }
I20260926 22:44:55.462879 21990 sys_catalog.cc:564] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:55.462888 21993 sys_catalog.cc:455] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.463097 21993 sys_catalog.cc:458] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: This master's current role is: LEARNER
I20260926 22:44:55.464378 22004 catalog_manager.cc:1259] Loaded cluster ID: 3494f7c568e34c0398ab343870a5edf0
I20260926 22:44:55.464423 22004 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: loading cluster ID for follower catalog manager: success
I20260926 22:44:55.465999 22004 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: acquiring CA information for follower catalog manager: success
I20260926 22:44:55.466269 22004 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 22:44:55.470108 21916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:55.470310 21916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:55.470358 21916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:55.471571 21916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:55.471632 21916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:55.471684 21916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:55.473146 21916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:41781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=32795
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:55.473407 21916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:55.473627 21916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:55.479350 22010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.479373 22012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.479346 22009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.479889 21916 server_base.cc:1034] running on GCE node
I20260926 22:44:55.480085 21916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:55.480325 21916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:55.481457 21916 hybrid_clock.cc:648] HybridClock initialized: now 1790462695481445 us; error 28 us; skew 500 ppm
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:55.483987 21916 init.cc:377] Logged in from keytab as oryx/127.14.184.193@KRBTEST.COM (short username oryx)
I20260926 22:44:55.484395 21916 webserver.cc:460] Webserver started at http://127.14.184.193:32795/ using document root <none> and password file <none>
I20260926 22:44:55.484628 21916 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.484700 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.485992 21916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:55.486550 22019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.486734 21916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:55.486812 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "1d02d950a5e04659a5cba4233efd171f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "2e2bfc4deb249196458f595f9dab7769"
server_key_iv: "8eaa1d89a59120cd17eb437596d823c5"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.487123 21916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.502767 21916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:55.504544 21916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:55.504648 21916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:55.504818 21916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:55.505122 21916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:55.505154 21916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.505177 21916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:55.505223 21916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.510766 21916 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:41781
I20260926 22:44:55.510828 22132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:41781 every 8 connection(s)
I20260926 22:44:55.511160 21916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:55.511686 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 21916
I20260926 22:44:55.511838 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21366
I20260926 22:44:55.516950 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:39213
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:43809
--webserver_port=44115
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.252@KRBTEST.COM
I20260926 22:44:55.524313 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.512522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:48095 (local address 127.14.184.253:34531)
0926 22:44:55.512599 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.512601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.514176 (+ 1575us) server_negotiation.cc:407] Connection header received
0926 22:44:55.514212 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.514214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.514226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.514293 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.515202 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.515921 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517567 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.517863 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517917 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.520758 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.520760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.520760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.520768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.522517 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.523067 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.523069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.523069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.523135 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.523468 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.523470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.523471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.523633 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.523690 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.524118 (+ 428us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.524197 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 22:44:55.524948 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.512095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.14.184.193:48095)
0926 22:44:55.514079 (+ 1984us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.514091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.514161 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.514343 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.514345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.515054 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.515109 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.515111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.516009 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.516010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.517479 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.517482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.517986 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.518048 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.518743 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.518749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.520656 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.522609 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.522615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.522617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.522968 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.523219 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.523220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.523220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.523350 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.523782 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.523789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.523963 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.524444 (+ 481us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.524619 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1965,"mutex_wait_us":540,"thread_start_us":51,"threads_started":1}
I20260926 22:44:55.525406 22135 heartbeater.cc:347] Connected to a master server at 127.14.184.253:34531
I20260926 22:44:55.525511 22135 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.525620 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.511821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:50439 (local address 127.14.184.254:43809)
0926 22:44:55.511878 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.511881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.512983 (+ 1102us) server_negotiation.cc:407] Connection header received
0926 22:44:55.513502 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.513503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.513517 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.513659 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.515056 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.515852 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517493 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.517773 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517813 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.521366 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.521375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.521375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.521382 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.523917 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.524286 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.524287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.524287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.524341 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.524790 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.524791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.524792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.524919 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.524965 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.525420 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.525527 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:55.525704 22135 heartbeater.cc:511] Master 127.14.184.253:34531 requested a full tablet report, sending...
I20260926 22:44:55.525990 22139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.511826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:50439)
0926 22:44:55.512860 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.512901 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.513390 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.513777 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.513781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.514636 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.514941 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.514947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.515983 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.515984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.517388 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.517391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.517901 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.518044 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.518880 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.518885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.521283 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.524001 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.524001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.524002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.524208 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.524642 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.524643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.524643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.524714 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.525233 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.525234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.525276 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.525767 (+ 491us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.525862 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"mutex_wait_us":31,"thread_start_us":100,"threads_started":1}
I20260926 22:44:55.526254 21769 ts_manager.cc:194] Registered new tserver with Master: 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781)
I20260926 22:44:55.526292 22134 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43809
I20260926 22:44:55.526335 22134 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.526414 22134 heartbeater.cc:511] Master 127.14.184.254:43809 requested a full tablet report, sending...
I20260926 22:44:55.526661 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.512613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:41133 (local address 127.14.184.252:40827)
0926 22:44:55.512737 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.512739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.514357 (+ 1618us) server_negotiation.cc:407] Connection header received
0926 22:44:55.514386 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.514387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.514399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.514459 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.515044 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.516158 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517491 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.517831 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517889 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.522554 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.522556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.522556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.522564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.524549 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.525057 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.525058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.525059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.525121 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.525323 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.525324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.525325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.526081 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.526144 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.526397 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.526534 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260926 22:44:55.526780 21170 ts_manager.cc:194] Re-registered known tserver with Master: 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781)
I20260926 22:44:55.526795 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.512210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.14.184.193:41133)
0926 22:44:55.514255 (+ 2045us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.514264 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.514304 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.514508 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.514509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.514748 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.514941 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.514947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.516256 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.516257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.517382 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.517386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.517944 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.517945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.518048 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.518705 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.518715 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.522448 (+ 3733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.524645 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.524646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.524647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.524972 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.525191 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.525192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.525192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.525250 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.526171 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.526172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.526210 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.526279 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.526371 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2019,"mutex_wait_us":5,"thread_start_us":51,"threads_started":1}
I20260926 22:44:55.527263 21170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:50439
I20260926 22:44:55.527527 22133 heartbeater.cc:347] Connected to a master server at 127.14.184.252:40827
I20260926 22:44:55.527575 22133 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.527637 22133 heartbeater.cc:511] Master 127.14.184.252:40827 requested a full tablet report, sending...
I20260926 22:44:55.527945 21954 ts_manager.cc:194] Registered new tserver with Master: 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781)
W20260926 22:44:55.600440 22141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:55.600632 22141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:55.600652 22141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:55.601779 22141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:55.601845 22141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:55.601888 22141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:55.603389 22141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:39213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=44115
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:55.603626 22141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:55.603845 22141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:55.608783 22149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.608809 22141 server_base.cc:1034] running on GCE node
W20260926 22:44:55.608814 22147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.608783 22146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.609134 22141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:55.609368 22141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:55.610558 22141 hybrid_clock.cc:648] HybridClock initialized: now 1790462695610550 us; error 36 us; skew 500 ppm
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:55.613026 22141 init.cc:377] Logged in from keytab as oryx/127.14.184.194@KRBTEST.COM (short username oryx)
I20260926 22:44:55.613472 22141 webserver.cc:460] Webserver started at http://127.14.184.194:44115/ using document root <none> and password file <none>
I20260926 22:44:55.613680 22141 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.613763 22141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.615150 22141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:55.615722 22156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.615891 22141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:55.615985 22141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "106191e21215457892ec751f5e401f76"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "eede75d405a6bde037af377c45c726cd"
server_key_iv: "842745e52c7d7a9746f8f36d85085291"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.616304 22141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.631801 22141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:55.633531 22141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:55.633687 22141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:55.633934 22141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:55.634217 22141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:55.634267 22141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.634298 22141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:55.634325 22141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.640077 22141 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:39213
I20260926 22:44:55.640132 22269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:39213 every 8 connection(s)
I20260926 22:44:55.640594 22141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:55.642400 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 22141
I20260926 22:44:55.642565 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21500
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:55.648730 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:45289
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:43809
--webserver_port=34019
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.252@KRBTEST.COM
I20260926 22:44:55.652452 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.641248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:38067 (local address 127.14.184.254:43809)
0926 22:44:55.641327 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.641329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.641586 (+ 257us) server_negotiation.cc:407] Connection header received
0926 22:44:55.642218 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.642220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.642233 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.642278 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.643355 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.643884 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.645038 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.645400 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.645474 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.648181 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.648183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.648183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.648190 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.650718 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651229 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.651230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.651231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.651296 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651607 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.651608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.651609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.651770 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.651840 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.652238 (+ 398us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.652330 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:44:55.652565 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.641619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:43073 (local address 127.14.184.252:40827)
0926 22:44:55.641660 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.641662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.641733 (+ 71us) server_negotiation.cc:407] Connection header received
0926 22:44:55.642231 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.642233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.642251 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.642285 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.644576 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.645258 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.646303 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.646712 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.646775 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.649266 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.649267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.649267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.649276 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.651277 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651716 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.651718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.651718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.651771 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651967 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.651967 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.651968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.652105 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.652153 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.652364 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.652456 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
I20260926 22:44:55.653049 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.641530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.14.184.194:43073)
0926 22:44:55.641654 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.641664 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.642126 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.642363 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.642364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.644387 (+ 2023us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.644475 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.644477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.645380 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.645381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.646214 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.646217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.646828 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.646829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.646911 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.647357 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.647362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.649156 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.651366 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.651367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.651619 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.651847 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651847 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.651848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.651898 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.652203 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.652204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.652250 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.652671 (+ 421us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.652850 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":25,"thread_start_us":82,"threads_started":1}
I20260926 22:44:55.653052 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.641292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:38067)
0926 22:44:55.641446 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.641480 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.642108 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.642354 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.642358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.643059 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.643260 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.643264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.643976 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.643977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.644928 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.644931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.645484 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.645485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.645634 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.646231 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.646240 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.648074 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.650791 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.650798 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.650801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.651135 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.651369 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.651370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.651370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.651523 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.651886 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.651892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.652051 (+ 159us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.652548 (+ 497us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.652755 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260926 22:44:55.653558 22271 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43809
I20260926 22:44:55.653568 22270 heartbeater.cc:347] Connected to a master server at 127.14.184.252:40827
I20260926 22:44:55.653666 22270 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.653664 22271 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.653887 22270 heartbeater.cc:511] Master 127.14.184.252:40827 requested a full tablet report, sending...
I20260926 22:44:55.653894 22271 heartbeater.cc:511] Master 127.14.184.254:43809 requested a full tablet report, sending...
I20260926 22:44:55.654274 21170 ts_manager.cc:194] Re-registered known tserver with Master: 106191e21215457892ec751f5e401f76 (127.14.184.194:39213)
I20260926 22:44:55.654623 21954 ts_manager.cc:194] Registered new tserver with Master: 106191e21215457892ec751f5e401f76 (127.14.184.194:39213)
I20260926 22:44:55.654858 21170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:38067
I20260926 22:44:55.656764 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.641915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:43233 (local address 127.14.184.253:34531)
0926 22:44:55.641979 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.641981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.641988 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:55.646860 (+ 4872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.646862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.646877 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.646915 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.647792 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.648502 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.649533 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.649844 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.649913 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.654036 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.654037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.654037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.654045 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.655646 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.656000 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.656001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.656001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.656060 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.656261 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.656262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.656263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.656375 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.656449 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.656595 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.656671 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 22:44:55.656972 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.641730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.14.184.194:43233)
0926 22:44:55.641825 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.641835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.642126 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.647422 (+ 5296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.647423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.647633 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.647693 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.647695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.648604 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.648605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.649443 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.649445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.650820 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.650821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.650896 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.651640 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.651645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.653889 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.655742 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.655743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.655743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.655913 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.656135 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.656136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.656136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.656198 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.656442 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.656443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.656483 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.656571 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.656673 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":58,"threads_started":1}
I20260926 22:44:55.657411 22272 heartbeater.cc:347] Connected to a master server at 127.14.184.253:34531
I20260926 22:44:55.657470 22272 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.657543 22272 heartbeater.cc:511] Master 127.14.184.253:34531 requested a full tablet report, sending...
I20260926 22:44:55.657841 21769 ts_manager.cc:194] Registered new tserver with Master: 106191e21215457892ec751f5e401f76 (127.14.184.194:39213)
W20260926 22:44:55.731983 22277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:55.732362 22277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:55.732419 22277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:55.733587 22277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:55.733625 22277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:55.733652 22277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:55.735191 22277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:45289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=34019
--tserver_master_addrs=127.14.184.254:43809,127.14.184.253:34531,127.14.184.252:40827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:55.735428 22277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:55.735677 22277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:55.740556 22285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:55.740556 22283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.740585 22277 server_base.cc:1034] running on GCE node
W20260926 22:44:55.740545 22282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:55.740940 22277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:55.741196 22277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:55.742451 22277 hybrid_clock.cc:648] HybridClock initialized: now 1790462695742435 us; error 44 us; skew 500 ppm
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:55.745069 22277 init.cc:377] Logged in from keytab as oryx/127.14.184.195@KRBTEST.COM (short username oryx)
I20260926 22:44:55.745550 22277 webserver.cc:460] Webserver started at http://127.14.184.195:34019/ using document root <none> and password file <none>
I20260926 22:44:55.745842 22277 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:55.745909 22277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:55.747273 22277 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:55.747822 22292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.747980 22277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.748080 22277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "a5d19e92a1a4434b80386a7678cc060f"
format_stamp: "Formatted at 2026-09-26 22:44:54 on dist-test-slave-3986"
server_key: "388502579a735ff09da4df343cecfe10"
server_key_iv: "bc5b8dbc83a0a3b92d2e3a590d4756e3"
server_key_version: "encryptionkey@0"
I20260926 22:44:55.748397 22277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:55.763718 22277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:55.766033 22277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:55.766188 22277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:55.766408 22277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:44:55.766805 22277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:55.766860 22277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.766899 22277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:55.766917 22277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:55.772792 22277 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:45289
I20260926 22:44:55.772840 22405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:45289 every 8 connection(s)
I20260926 22:44:55.773159 22277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:55.774051 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 22277
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.253@KRBTEST.COM
Sep 26 22:44:55 dist-test-slave-3986 krb5kdc[21125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462695, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.252@KRBTEST.COM
I20260926 22:44:55.786650 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.774436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:58829 (local address 127.14.184.254:43809)
0926 22:44:55.774565 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.774567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.774576 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:55.775675 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.775676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.775688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.775737 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.776923 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.777631 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.779794 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780092 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780153 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.783237 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.783238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.783239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.783245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.784789 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785530 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.785531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.785532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.785596 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785887 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.785888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.785889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.786013 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.786073 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.786484 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.786559 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 22:44:55.786876 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.774283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.14.184.195:33391)
0926 22:44:55.774390 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.774399 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.775512 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.775868 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.775872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.776420 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.776691 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.776697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.777680 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.777681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.778899 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.778902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780881 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.780944 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.781396 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.781400 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.783470 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.785289 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.785291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.785560 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.785752 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.785753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.785815 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.786101 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.786107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.786260 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.786400 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.786593 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260926 22:44:55.787190 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.774128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:58829)
0926 22:44:55.774229 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.774238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.775514 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.775868 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.775872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.776422 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.776841 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.776842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.777731 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.777732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.779690 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.779693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780656 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.780718 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.781236 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.781243 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.783090 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.784879 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.784886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.784888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.785452 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.785664 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.785665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.785815 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.786101 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.786107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.786260 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.786380 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.786851 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":1704,"thread_start_us":58,"threads_started":1}
I20260926 22:44:55.787408 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.774525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:33391 (local address 127.14.184.253:34531)
0926 22:44:55.774609 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.774611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.774626 (+ 15us) server_negotiation.cc:407] Connection header received
0926 22:44:55.775678 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.775679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.775689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.775737 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.776775 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.777542 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780202 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780538 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780603 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.783538 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.783539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.783539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.783546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.785206 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785633 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.785634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.785635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.785687 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785893 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.785900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.785901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.786013 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.786073 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.786608 (+ 535us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.786702 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 22:44:55.787516 22407 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43809
I20260926 22:44:55.787606 22407 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.787788 22407 heartbeater.cc:511] Master 127.14.184.254:43809 requested a full tablet report, sending...
I20260926 22:44:55.787956 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.775229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.14.184.254:43809)
0926 22:44:55.775366 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.775368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.775397 (+ 29us) server_negotiation.cc:407] Connection header received
0926 22:44:55.775468 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.775469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.775481 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.775559 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.776446 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.777225 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.778292 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.778587 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.778640 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.780872 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.780873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.780874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.780880 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.783261 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.785096 (+ 1835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.785098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.785098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.786925 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.787202 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.787203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.787203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.787334 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.787431 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.787787 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.787866 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260926 22:44:55.788067 22408 heartbeater.cc:347] Connected to a master server at 127.14.184.253:34531
I20260926 22:44:55.788125 22408 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.788193 22408 heartbeater.cc:511] Master 127.14.184.253:34531 requested a full tablet report, sending...
I20260926 22:44:55.788403 21170 ts_manager.cc:194] Re-registered known tserver with Master: a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289)
I20260926 22:44:55.788555 21769 ts_manager.cc:194] Registered new tserver with Master: a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289)
I20260926 22:44:55.788856 21170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:58829
I20260926 22:44:55.790292 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.775566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37904 (local address 127.14.184.253:34531)
0926 22:44:55.777332 (+ 1766us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.777334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.777340 (+ 6us) server_negotiation.cc:407] Connection header received
0926 22:44:55.777360 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.777361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.777373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.778158 (+ 785us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.778194 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.778923 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780601 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780909 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.781202 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.782641 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.782642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.782642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.782649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.786411 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.786865 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.786866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.786866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.786935 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.787183 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.787184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.787185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.789631 (+ 2446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.789740 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.789926 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.790164 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1747,"thread_start_us":73,"threads_started":1}
I20260926 22:44:55.791663 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.773853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:51823 (local address 127.14.184.252:40827)
0926 22:44:55.773925 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.773928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.774180 (+ 252us) server_negotiation.cc:407] Connection header received
0926 22:44:55.775721 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.775722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.775735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.775765 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.778130 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.778870 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780156 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780443 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780479 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.787517 (+ 7038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.787518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.787519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.787525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.789442 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.790418 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.790420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.790420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.790481 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.790741 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.790742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.790742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.791134 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.791180 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.791471 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.791580 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 22:44:55.791872 22410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.773875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.14.184.195:51823)
0926 22:44:55.774040 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.774086 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.775634 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.775869 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.775873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.777914 (+ 2041us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:55.778016 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.778018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.779216 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.779217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.780072 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.780074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780558 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.780559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.780685 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.784207 (+ 3522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:55.784212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:55.787427 (+ 3215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:55.789514 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.789514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.789515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.789980 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.790564 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:55.790565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:55.790565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:55.790633 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:55.791251 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:55.791252 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:55.791295 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.791418 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.791551 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260926 22:44:55.792312 22406 heartbeater.cc:347] Connected to a master server at 127.14.184.252:40827
I20260926 22:44:55.792364 22406 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:55.792433 22406 heartbeater.cc:511] Master 127.14.184.252:40827 requested a full tablet report, sending...
I20260926 22:44:55.792711 21954 ts_manager.cc:194] Registered new tserver with Master: a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289)
I20260926 22:44:55.790609 21170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:55.792951 21170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:55.793267 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.776569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.14.184.252:40827)
0926 22:44:55.779030 (+ 2461us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.779032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.779038 (+ 6us) server_negotiation.cc:407] Connection header received
0926 22:44:55.779057 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.779058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.779069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.779125 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:55.779825 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.780523 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.783683 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.784018 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.784084 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.789776 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:55.789777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:55.789778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:55.789784 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:55.791837 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.792268 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.792269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.792269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.792351 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:55.792661 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:55.792662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:55.792662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:55.792789 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:55.792883 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.793068 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.793154 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2444,"thread_start_us":58,"threads_started":1}
I20260926 22:44:55.800165 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.796513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:39213 (local address 127.0.0.1:51014)
0926 22:44:55.796627 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.796636 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.796837 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.797154 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.797155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.797157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.797201 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.797202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.798063 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.798064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.799042 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.799045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.799932 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.799933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.799990 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.799994 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.800044 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.800086 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":82,"threads_started":1}
I20260926 22:44:55.800170 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.796735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.14.184.194:39213)
0926 22:44:55.796918 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.796921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.796931 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:55.796989 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.796991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.797008 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.797086 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.797257 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.797978 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.799134 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.799833 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.799949 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.800007 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.800060 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.800096 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260926 22:44:55.800802 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.796433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:41781 (local address 127.0.0.1:37620)
0926 22:44:55.796467 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.796478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.796515 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.797299 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.797300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.797301 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.797327 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.797328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.798535 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.798536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.799463 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.799465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.800593 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.800594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.800657 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.800661 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.800691 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.800734 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 22:44:55.800952 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.796525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:45289 (local address 127.0.0.1:38716)
0926 22:44:55.796980 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.796990 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.797037 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.797250 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.797251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.797253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.797301 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.797302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.798511 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.798512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.799626 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.799628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.800717 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.800718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.800787 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.800791 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.800852 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.800889 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":34,"thread_start_us":87,"threads_started":1}
I20260926 22:44:55.800817 22204 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6a236941b74507a7a0391bcd5e0c25 (DEFAULT_TABLE table=test-table [id=1e343d1e0846444a846243257484330b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:55.801198 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6a236941b74507a7a0391bcd5e0c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:55.801280 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.796704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.14.184.195:45289)
0926 22:44:55.796869 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.796874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.797036 (+ 162us) server_negotiation.cc:407] Connection header received
0926 22:44:55.797102 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.797105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.797121 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.797194 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.797384 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.798425 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.799705 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.800637 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.800723 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.800772 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.801137 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.801180 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260926 22:44:55.801494 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.796520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.14.184.193:41781)
0926 22:44:55.797165 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.797170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.797179 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:55.797213 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.797215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.797230 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.797305 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.797385 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.798426 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.799550 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.800495 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.801001 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.801081 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.801375 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.801419 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":95,"threads_started":1}
I20260926 22:44:55.801867 22340 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6a236941b74507a7a0391bcd5e0c25 (DEFAULT_TABLE table=test-table [id=1e343d1e0846444a846243257484330b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:55.802047 22067 tablet_service.cc:1467] Processing CreateTablet for tablet 8b6a236941b74507a7a0391bcd5e0c25 (DEFAULT_TABLE table=test-table [id=1e343d1e0846444a846243257484330b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:55.802186 22340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6a236941b74507a7a0391bcd5e0c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:55.802277 22067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b6a236941b74507a7a0391bcd5e0c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:55.804844 22432 tablet_bootstrap.cc:492] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Bootstrap starting.
I20260926 22:44:55.805469 22433 tablet_bootstrap.cc:492] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Bootstrap starting.
I20260926 22:44:55.805650 22432 tablet_bootstrap.cc:654] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:55.805729 22434 tablet_bootstrap.cc:492] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Bootstrap starting.
I20260926 22:44:55.805989 22432 log.cc:824] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:55.806113 22433 tablet_bootstrap.cc:654] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:55.806339 22434 tablet_bootstrap.cc:654] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:55.806440 22433 log.cc:824] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:55.806620 22434 log.cc:824] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:55.806933 22432 tablet_bootstrap.cc:492] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: No bootstrap required, opened a new log
I20260926 22:44:55.807027 22432 ts_tablet_manager.cc:1397] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:55.807372 22434 tablet_bootstrap.cc:492] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: No bootstrap required, opened a new log
I20260926 22:44:55.807439 22434 ts_tablet_manager.cc:1397] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 22:44:55.809038 22432 raft_consensus.cc:348] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.809190 22432 raft_consensus.cc:374] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:55.809238 22432 raft_consensus.cc:729] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106191e21215457892ec751f5e401f76, State: Initialized, Role: FOLLOWER
I20260926 22:44:55.809346 22432 consensus_queue.cc:260] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.809283 22434 raft_consensus.cc:348] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.809445 22434 raft_consensus.cc:374] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:55.809489 22434 raft_consensus.cc:729] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d19e92a1a4434b80386a7678cc060f, State: Initialized, Role: FOLLOWER
I20260926 22:44:55.809600 22432 ts_tablet_manager.cc:1428] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:44:55.809594 22434 consensus_queue.cc:260] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.809684 22271 heartbeater.cc:503] Master 127.14.184.254:43809 was elected leader, sending a full tablet report...
I20260926 22:44:55.809823 22434 ts_tablet_manager.cc:1428] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 22:44:55.810103 22407 heartbeater.cc:503] Master 127.14.184.254:43809 was elected leader, sending a full tablet report...
I20260926 22:44:55.810701 22433 tablet_bootstrap.cc:492] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: No bootstrap required, opened a new log
I20260926 22:44:55.810772 22433 ts_tablet_manager.cc:1397] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 22:44:55.812644 22433 raft_consensus.cc:348] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.812763 22433 raft_consensus.cc:374] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:55.812829 22433 raft_consensus.cc:729] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d02d950a5e04659a5cba4233efd171f, State: Initialized, Role: FOLLOWER
I20260926 22:44:55.812943 22433 consensus_queue.cc:260] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.813246 22433 ts_tablet_manager.cc:1428] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 22:44:55.813268 22134 heartbeater.cc:503] Master 127.14.184.254:43809 was elected leader, sending a full tablet report...
W20260926 22:44:55.891294 22273 tablet.cc:2369] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:55.936218 22439 raft_consensus.cc:482] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:55.936334 22439 raft_consensus.cc:504] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.936633 22439 leader_election.cc:288] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781), 106191e21215457892ec751f5e401f76 (127.14.184.194:39213)
I20260926 22:44:55.939390 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.936929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:49621 (local address 127.14.184.194:39213)
0926 22:44:55.936996 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.936999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.937009 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:55.937041 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.937042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.937054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.937117 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.937314 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.937773 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.938692 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.939055 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.939135 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.939143 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.939295 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.939329 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 22:44:55.939390 22410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.936806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:39213 (local address 127.14.184.195:49621)
0926 22:44:55.936870 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.936883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.936945 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.937125 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.937126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.937128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.937224 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.937226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.937853 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.937855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.938574 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.938576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.939131 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.939132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.939212 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.939216 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.939287 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.939323 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 22:44:55.939608 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.936816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:41781 (local address 127.14.184.195:58889)
0926 22:44:55.937209 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.937219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.937268 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.937421 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.937422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.937423 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.937475 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.937476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.938088 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.938089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.938835 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.938838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.939364 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.939365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.939438 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.939441 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.939494 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.939540 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":319}
I20260926 22:44:55.939857 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.936932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:58889 (local address 127.14.184.193:41781)
0926 22:44:55.937008 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.937011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.937277 (+ 266us) server_negotiation.cc:407] Connection header received
0926 22:44:55.937332 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.937334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.937343 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.937421 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.937549 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.938003 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.938922 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.939286 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.939366 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.939380 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.939750 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.939794 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 22:44:55.939849 22224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a236941b74507a7a0391bcd5e0c25" candidate_uuid: "a5d19e92a1a4434b80386a7678cc060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106191e21215457892ec751f5e401f76" is_pre_election: true
I20260926 22:44:55.940013 22224 raft_consensus.cc:2454] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5d19e92a1a4434b80386a7678cc060f in term 0.
I20260926 22:44:55.940236 22295 leader_election.cc:302] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106191e21215457892ec751f5e401f76, a5d19e92a1a4434b80386a7678cc060f; no voters:
I20260926 22:44:55.940245 22087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a236941b74507a7a0391bcd5e0c25" candidate_uuid: "a5d19e92a1a4434b80386a7678cc060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d02d950a5e04659a5cba4233efd171f" is_pre_election: true
I20260926 22:44:55.940363 22087 raft_consensus.cc:2454] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5d19e92a1a4434b80386a7678cc060f in term 0.
I20260926 22:44:55.940375 22439 raft_consensus.cc:2781] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:55.940438 22439 raft_consensus.cc:482] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:55.940472 22439 raft_consensus.cc:3037] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:55.941147 22439 raft_consensus.cc:504] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.941356 22439 leader_election.cc:288] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [CANDIDATE]: Term 1 election: Requested vote from peers 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781), 106191e21215457892ec751f5e401f76 (127.14.184.194:39213)
I20260926 22:44:55.941437 22087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a236941b74507a7a0391bcd5e0c25" candidate_uuid: "a5d19e92a1a4434b80386a7678cc060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d02d950a5e04659a5cba4233efd171f"
I20260926 22:44:55.941490 22224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b6a236941b74507a7a0391bcd5e0c25" candidate_uuid: "a5d19e92a1a4434b80386a7678cc060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106191e21215457892ec751f5e401f76"
I20260926 22:44:55.941524 22087 raft_consensus.cc:3037] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:55.941560 22224 raft_consensus.cc:3037] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:55.942368 22224 raft_consensus.cc:2454] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5d19e92a1a4434b80386a7678cc060f in term 1.
I20260926 22:44:55.942378 22087 raft_consensus.cc:2454] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5d19e92a1a4434b80386a7678cc060f in term 1.
I20260926 22:44:55.942531 22293 leader_election.cc:302] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d02d950a5e04659a5cba4233efd171f, a5d19e92a1a4434b80386a7678cc060f; no voters:
I20260926 22:44:55.942617 22439 raft_consensus.cc:2781] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:55.942775 22439 raft_consensus.cc:686] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 1 LEADER]: Becoming Leader. State: Replica: a5d19e92a1a4434b80386a7678cc060f, State: Running, Role: LEADER
I20260926 22:44:55.942870 22439 consensus_queue.cc:237] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } }
I20260926 22:44:55.944204 21170 catalog_manager.cc:5495] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f reported cstate change: term changed from 0 to 1, leader changed from <none> to a5d19e92a1a4434b80386a7678cc060f (127.14.184.195). New cstate: current_term: 1 leader_uuid: "a5d19e92a1a4434b80386a7678cc060f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:55.946846 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.943970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.14.184.252:40827)
0926 22:44:55.944010 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.944012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.944033 (+ 21us) server_negotiation.cc:407] Connection header received
0926 22:44:55.944096 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.944097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.944116 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.944173 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:55.944330 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.944990 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.946104 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.946512 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.946596 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.946665 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.946742 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.946783 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:55.946864 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.943910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:39592)
0926 22:44:55.943968 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:55.943980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:55.944037 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:55.944213 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:55.944214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:55.944215 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:55.944259 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.944260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.945061 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.945062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.946020 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:55.946023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.946593 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.946594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:55.946651 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.946654 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:55.946735 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 22:44:55.946783 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 22:44:55.947153 21964 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260926 22:44:55.947993 21993 sys_catalog.cc:455] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c988e7c94b20406e9783b933f740c88e. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.948086 21993 sys_catalog.cc:458] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: This master's current role is: LEARNER
I20260926 22:44:55.948427 21731 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: NON_VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: true } } }
I20260926 22:44:55.948570 21731 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.949005 22446 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: attempting to promote NON_VOTER f088109a197045c68e1552445490331b to VOTER
I20260926 22:44:55.949113 22446 consensus_queue.cc:237] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } }
I20260926 22:44:55.949488 21964 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 LEARNER]: Refusing update from remote peer c988e7c94b20406e9783b933f740c88e: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 22:44:55.949548 21779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Refusing update from remote peer c988e7c94b20406e9783b933f740c88e: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 22:44:55.949674 21729 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260926 22:44:55.949754 21731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260926 22:44:55.950297 21729 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f088109a197045c68e1552445490331b (127.14.184.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.950493 21779 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f088109a197045c68e1552445490331b (127.14.184.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.950536 21964 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, f088109a197045c68e1552445490331b (127.14.184.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.951258 21729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.951335 21729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.951395 21729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.951465 21729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.951423 22448 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.951531 21729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.951562 22448 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:44:55.951586 21729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:55.951576 21993 sys_catalog.cc:455] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c988e7c94b20406e9783b933f740c88e" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:55.951646 21993 sys_catalog.cc:458] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:44:55.981397 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.979358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.14.184.195:45289)
0926 22:44:55.979417 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.979423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:55.979431 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:55.979465 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:55.979466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:55.979482 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:55.979545 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:55.979664 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.980119 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.980774 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:55.980947 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:55.980995 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:55.981121 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:55.981215 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:55.981272 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:55.981304 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 22:44:55.981320 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 22:44:55.983116 22087 raft_consensus.cc:1261] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 1 FOLLOWER]: Refusing update from remote peer a5d19e92a1a4434b80386a7678cc060f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:55.983145 22224 raft_consensus.cc:1261] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Refusing update from remote peer a5d19e92a1a4434b80386a7678cc060f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:55.983462 22439 consensus_queue.cc:1035] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:55.983604 22441 consensus_queue.cc:1035] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [LEADER]: Connected to new peer: Peer: permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:55.986040 22450 mvcc.cc:205] Tried to move back new op lower bound from 7333735202744176640 to 7333735202583023616. Current Snapshot: MvccSnapshot[applied={T|T < 7333735202744176640}]
I20260926 22:44:55.986332 22451 mvcc.cc:205] Tried to move back new op lower bound from 7333735202744176640 to 7333735202583023616. Current Snapshot: MvccSnapshot[applied={T|T < 7333735202744176640}]
I20260926 22:44:55.989444 21169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:55.989540 21169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:55.990836 21169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c988e7c94b20406e9783b933f740c88e: Sending DeleteTablet for 3 replicas of tablet 8b6a236941b74507a7a0391bcd5e0c25
I20260926 22:44:55.991210 22204 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b6a236941b74507a7a0391bcd5e0c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:55 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:51014
I20260926 22:44:55.991310 22340 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b6a236941b74507a7a0391bcd5e0c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:55 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38716
I20260926 22:44:55.991590 22067 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b6a236941b74507a7a0391bcd5e0c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:55 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:37620
I20260926 22:44:55.991784 22461 tablet_replica.cc:330] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: stopping tablet replica
I20260926 22:44:55.991791 22462 tablet_replica.cc:330] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: stopping tablet replica
I20260926 22:44:55.991933 22461 raft_consensus.cc:2229] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:55.991948 22462 raft_consensus.cc:2229] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:55.992065 22461 raft_consensus.cc:2258] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:55.992174 22462 raft_consensus.cc:2258] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:55.992491 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21138
I20260926 22:44:55.992627 22461 ts_tablet_manager.cc:1905] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:55.993117 22462 ts_tablet_manager.cc:1905] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:55.994341 22461 ts_tablet_manager.cc:1918] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:55.994475 22461 log.cc:1196] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/8b6a236941b74507a7a0391bcd5e0c25
I20260926 22:44:55.994812 22462 ts_tablet_manager.cc:1918] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:55.994894 22462 log.cc:1196] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/8b6a236941b74507a7a0391bcd5e0c25
I20260926 22:44:55.994964 22461 ts_tablet_manager.cc:1939] T 8b6a236941b74507a7a0391bcd5e0c25 P a5d19e92a1a4434b80386a7678cc060f: Deleting consensus metadata
I20260926 22:44:55.995157 22462 ts_tablet_manager.cc:1939] T 8b6a236941b74507a7a0391bcd5e0c25 P 1d02d950a5e04659a5cba4233efd171f: Deleting consensus metadata
W20260926 22:44:55.998754 22296 connection.cc:535] server connection from 127.0.0.1:38716 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 22:44:55.998916 22020 connection.cc:535] server connection from 127.0.0.1:37620 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 22:44:55.999625 22460 tablet_replica.cc:330] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: stopping tablet replica
I20260926 22:44:55.999732 22460 raft_consensus.cc:2229] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:55.999838 22460 raft_consensus.cc:2258] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:56.000299 22460 ts_tablet_manager.cc:1905] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:56.001924 22460 ts_tablet_manager.cc:1918] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:56.002005 22460 log.cc:1196] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/8b6a236941b74507a7a0391bcd5e0c25
I20260926 22:44:56.002231 22460 ts_tablet_manager.cc:1939] T 8b6a236941b74507a7a0391bcd5e0c25 P 106191e21215457892ec751f5e401f76: Deleting consensus metadata
W20260926 22:44:56.002854 22160 connection.cc:441] server connection from 127.0.0.1:51014 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51014 (request call id 1) could send its response
I20260926 22:44:56.011642 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:56.003349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.14.184.252:40827)
0926 22:44:56.003390 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:56.003392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:56.003753 (+ 361us) server_negotiation.cc:407] Connection header received
0926 22:44:56.003857 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:56.003858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:56.003879 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:56.003913 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:56.004569 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:56.005278 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:56.006338 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:56.006626 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:56.006696 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:56.008269 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:56.008271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:56.008271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:56.008278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:56.010392 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:56.010834 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:56.010836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:56.010836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:56.010899 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:56.011098 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:56.011099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:56.011099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:56.011217 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:56.011285 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:56.011483 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 22:44:56.011551 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:56.011642 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:56.003131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.14.184.253:34531)
0926 22:44:56.003177 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:56.003178 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:56.003281 (+ 103us) server_negotiation.cc:407] Connection header received
0926 22:44:56.003382 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:56.003383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:56.003398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:56.003430 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:56.004190 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:56.004965 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:56.005964 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:56.006270 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:56.006326 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:56.007852 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:56.007853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:56.007854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:56.007861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:56.010181 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:56.010584 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:56.010585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:56.010586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:56.010642 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:56.010890 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:56.010891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:56.010891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:56.011030 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:56.011122 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:56.011450 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 22:44:56.011536 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:44:56.011963 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:55.991941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.14.184.252:40827)
0926 22:44:55.991981 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:55.991983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:56.002837 (+ 10854us) server_negotiation.cc:407] Connection header received
0926 22:44:56.002965 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:56.002966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:56.002978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:56.003041 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:56.003844 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:56.004618 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:56.005731 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:56.006040 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:56.006079 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:56.008115 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:56.008117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:56.008117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:56.008124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:56.010461 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:56.010953 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:56.010954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:56.010954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:56.011008 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:56.011194 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:56.011195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:56.011195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:56.011308 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:56.011383 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:56.011581 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 22:44:56.011661 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:44:56.987548 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:56.987247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:44691)
0926 22:44:56.987426 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:56.987502 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260926 22:44:56.987669 22471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:56.987433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:54863)
0926 22:44:56.987546 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:56.987616 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
W20260926 22:44:56.988149 22134 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43809 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
I20260926 22:44:56.988241 22472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:56.987994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:53511)
0926 22:44:56.988133 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:56.988200 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260926 22:44:56.988394 22271 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43809 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
W20260926 22:44:56.988680 22407 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43809 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
I20260926 22:44:57.491556 22473 raft_consensus.cc:482] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c988e7c94b20406e9783b933f740c88e)
I20260926 22:44:57.491674 22473 raft_consensus.cc:504] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } }
I20260926 22:44:57.491799 22474 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c988e7c94b20406e9783b933f740c88e)
I20260926 22:44:57.491906 22473 leader_election.cc:288] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809), f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253:34531)
I20260926 22:44:57.491953 22474 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } }
I20260926 22:44:57.492183 22474 leader_election.cc:288] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809), f088109a197045c68e1552445490331b (127.14.184.252:40827)
I20260926 22:44:57.492480 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.492025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43222)
0926 22:44:57.492357 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.492432 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":37,"threads_started":1}
I20260926 22:44:57.492580 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.492278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43226)
0926 22:44:57.492481 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.492536 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":39,"threads_started":1}
W20260926 22:44:57.492609 21940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
W20260926 22:44:57.492684 21755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
I20260926 22:44:57.493175 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.493085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43248)
0926 22:44:57.493122 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.493150 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260926 22:44:57.493266 21755 leader_election.cc:334] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809): Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
I20260926 22:44:57.493191 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.493051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43242)
0926 22:44:57.493131 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.493159 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 22:44:57.493409 21940 leader_election.cc:334] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809): Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
I20260926 22:44:57.496106 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.492146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:34531 (local address 127.0.0.1:43786)
0926 22:44:57.492418 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.492430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.492651 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.492926 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.492927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.492930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.492987 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.492989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.493780 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.493781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.495018 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.495021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.495877 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.495878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.495941 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.495944 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.495999 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.496041 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":50,"threads_started":1}
I20260926 22:44:57.496124 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.492240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.14.184.253:34531)
0926 22:44:57.492360 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.492363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.492516 (+ 153us) server_negotiation.cc:407] Connection header received
0926 22:44:57.492800 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.492801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.492815 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.492871 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.493070 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.493704 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.495108 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.495779 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.495861 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.495870 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.496019 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.496067 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260926 22:44:57.496336 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.492402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:40827 (local address 127.0.0.1:34064)
0926 22:44:57.492831 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.492842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.492925 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.493421 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.493422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.493425 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.493476 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.493478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.494369 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.494370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.495401 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.495404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.495956 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.495957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.496050 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.496053 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.496116 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.496161 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":34,"threads_started":1}
I20260926 22:44:57.496343 21779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f088109a197045c68e1552445490331b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" is_pre_election: true
I20260926 22:44:57.496467 21779 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f088109a197045c68e1552445490331b in term 1.
I20260926 22:44:57.496618 21940 leader_election.cc:302] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f088109a197045c68e1552445490331b, f6f853b10e6a4ec4b69d8bec2a7d927d; no voters: c988e7c94b20406e9783b933f740c88e
I20260926 22:44:57.496719 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.492506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34064 (local address 127.14.184.252:40827)
0926 22:44:57.492581 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.492583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.492906 (+ 323us) server_negotiation.cc:407] Connection header received
0926 22:44:57.492994 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.492995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.493005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.493071 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.493541 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.494282 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.495483 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.495883 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.495943 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.495950 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.496122 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.496618 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"thread_start_us":38,"threads_started":1}
I20260926 22:44:57.496747 22473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 22:44:57.496778 22473 raft_consensus.cc:482] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c988e7c94b20406e9783b933f740c88e)
I20260926 22:44:57.496809 22473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:44:57.496878 21964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f088109a197045c68e1552445490331b" is_pre_election: true
I20260926 22:44:57.497530 22473 raft_consensus.cc:504] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } }
I20260926 22:44:57.497655 21964 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f6f853b10e6a4ec4b69d8bec2a7d927d in current term 2: Already voted for candidate f088109a197045c68e1552445490331b in this term.
I20260926 22:44:57.497663 22473 leader_election.cc:288] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 2 election: Requested vote from peers c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809), f6f853b10e6a4ec4b69d8bec2a7d927d (127.14.184.253:34531)
I20260926 22:44:57.497893 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.497738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43262)
0926 22:44:57.497816 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.497844 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 22:44:57.497901 21755 leader_election.cc:302] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6f853b10e6a4ec4b69d8bec2a7d927d; no voters: c988e7c94b20406e9783b933f740c88e, f088109a197045c68e1552445490331b
I20260926 22:44:57.497911 21779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f088109a197045c68e1552445490331b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d"
I20260926 22:44:57.497996 21779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:44:57.498242 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.498146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43274)
0926 22:44:57.498194 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.498222 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 22:44:57.498310 21940 leader_election.cc:334] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809): Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
I20260926 22:44:57.498715 21779 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f088109a197045c68e1552445490331b in term 2.
I20260926 22:44:57.498798 22474 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 22:44:57.498874 21940 leader_election.cc:302] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f088109a197045c68e1552445490331b, f6f853b10e6a4ec4b69d8bec2a7d927d; no voters: c988e7c94b20406e9783b933f740c88e
I20260926 22:44:57.498956 22473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:44:57.499114 22473 raft_consensus.cc:686] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [term 2 LEADER]: Becoming Leader. State: Replica: f088109a197045c68e1552445490331b, State: Running, Role: LEADER
I20260926 22:44:57.499207 22473 consensus_queue.cc:237] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } }
I20260926 22:44:57.499579 22483 sys_catalog.cc:455] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f088109a197045c68e1552445490331b. Latest consensus state: current_term: 2 leader_uuid: "f088109a197045c68e1552445490331b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:57.499675 22483 sys_catalog.cc:458] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:57.499866 22485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:57.500347 22485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:57.500437 22485 catalog_manager.cc:1259] Loaded cluster ID: 3494f7c568e34c0398ab343870a5edf0
I20260926 22:44:57.500473 22485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:44:57.500638 22485 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:57.500720 22485 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Loaded TSK: 0
I20260926 22:44:57.500862 22485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:44:57.556231 21954 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:57.557849 21954 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:57.558868 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.558766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43286)
0926 22:44:57.558812 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.558844 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 22:44:57.558991 21779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [term 2 FOLLOWER]: Refusing update from remote peer f088109a197045c68e1552445490331b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 22:44:57.559248 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.559154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.0.0.1:43300)
0926 22:44:57.559208 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.559232 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 22:44:57.559242 22473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 22:44:57.559371 21940 consensus_peers.cc:487] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b -> Peer c988e7c94b20406e9783b933f740c88e (127.14.184.254:43809): Couldn't send request to peer c988e7c94b20406e9783b933f740c88e. Status: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 22:44:57.559867 22474 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f088109a197045c68e1552445490331b. Latest consensus state: current_term: 2 leader_uuid: "f088109a197045c68e1552445490331b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:57.559978 22474 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:44:57.560364 22483 sys_catalog.cc:455] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f088109a197045c68e1552445490331b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:57.560458 22483 sys_catalog.cc:458] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:57.560536 22474 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f088109a197045c68e1552445490331b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c988e7c94b20406e9783b933f740c88e" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43809 } } peers { permanent_uuid: "f6f853b10e6a4ec4b69d8bec2a7d927d" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 34531 } attrs { promote: false } } peers { permanent_uuid: "f088109a197045c68e1552445490331b" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 40827 } attrs { promote: false } } }
I20260926 22:44:57.560621 22474 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f853b10e6a4ec4b69d8bec2a7d927d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:44:57.565832 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.562305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:39213 (local address 127.0.0.1:58332)
0926 22:44:57.562348 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.562358 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.562405 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.562624 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.562625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.562628 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.562710 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.562712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.563772 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.563773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.564785 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.564787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.565566 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.565628 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.565631 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.565667 (+ 36us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.565697 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 22:44:57.565877 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.562553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.14.184.195:45289)
0926 22:44:57.562652 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.562654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.562661 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:57.562701 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.562702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.562711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.562768 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.562870 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.563563 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.564952 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.565494 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565566 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.565576 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.565756 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.565808 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":58,"threads_started":1}
I20260926 22:44:57.566054 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.562234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:41781 (local address 127.0.0.1:49034)
0926 22:44:57.562263 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.562274 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.562311 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.563087 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.563088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.563089 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.563134 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.563135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.564030 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.564031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.565048 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.565049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.565596 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.565909 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.565914 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.565961 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.566004 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":244}
I20260926 22:44:57.566103 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.562518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.14.184.193:41781)
0926 22:44:57.562959 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.562961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.562968 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:57.562992 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.562993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.563002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.563204 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.563240 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.563973 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565123 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.565501 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565622 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.565631 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.565980 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.566025 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":61,"threads_started":1}
I20260926 22:44:57.566222 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.562328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:45289 (local address 127.0.0.1:46300)
0926 22:44:57.562504 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.562513 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.562557 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.562770 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.562771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.562773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.562817 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.562819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.563659 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.563660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.564862 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.564865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.565593 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.565674 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.565678 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.565733 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.565764 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260926 22:44:57.566329 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.562368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.14.184.194:39213)
0926 22:44:57.562485 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.562487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.562495 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:57.562521 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.562521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.562533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.562783 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.562860 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.563676 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.564881 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.565470 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.565817 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.565825 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.565842 (+ 17us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.565877 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260926 22:44:57.566345 22067 tablet_service.cc:1467] Processing CreateTablet for tablet c94a8e3b389044f9a689f75bdad142b8 (DEFAULT_TABLE table=test-table [id=1922d66fa8c946849e0183e5e9389a11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:57.566479 22067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c94a8e3b389044f9a689f75bdad142b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:57.566555 22204 tablet_service.cc:1467] Processing CreateTablet for tablet c94a8e3b389044f9a689f75bdad142b8 (DEFAULT_TABLE table=test-table [id=1922d66fa8c946849e0183e5e9389a11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:57.566653 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c94a8e3b389044f9a689f75bdad142b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:57.566773 22340 tablet_service.cc:1467] Processing CreateTablet for tablet c94a8e3b389044f9a689f75bdad142b8 (DEFAULT_TABLE table=test-table [id=1922d66fa8c946849e0183e5e9389a11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:57.566885 22340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c94a8e3b389044f9a689f75bdad142b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:57.567585 22495 tablet_bootstrap.cc:492] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f: Bootstrap starting.
I20260926 22:44:57.567993 22495 tablet_bootstrap.cc:654] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:57.568475 22496 tablet_bootstrap.cc:492] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76: Bootstrap starting.
I20260926 22:44:57.568622 22495 tablet_bootstrap.cc:492] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f: No bootstrap required, opened a new log
I20260926 22:44:57.568674 22495 ts_tablet_manager.cc:1397] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:57.568828 22496 tablet_bootstrap.cc:654] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:57.568818 22495 raft_consensus.cc:348] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.568892 22495 raft_consensus.cc:374] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:57.568917 22495 raft_consensus.cc:729] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d02d950a5e04659a5cba4233efd171f, State: Initialized, Role: FOLLOWER
I20260926 22:44:57.568998 22495 consensus_queue.cc:260] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.569145 22497 tablet_bootstrap.cc:492] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Bootstrap starting.
I20260926 22:44:57.569156 22495 ts_tablet_manager.cc:1428] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:57.569658 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.569307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:46097)
0926 22:44:57.569433 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.569488 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":89,"threads_started":1}
I20260926 22:44:57.569826 22133 heartbeater.cc:503] Master 127.14.184.252:40827 was elected leader, sending a full tablet report...
I20260926 22:44:57.569835 22497 tablet_bootstrap.cc:654] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:57.570062 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.569969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:34521)
0926 22:44:57.570013 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.570039 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 22:44:57.570254 22496 tablet_bootstrap.cc:492] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76: No bootstrap required, opened a new log
I20260926 22:44:57.570307 22496 ts_tablet_manager.cc:1397] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:57.570421 22496 raft_consensus.cc:348] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.570506 22496 raft_consensus.cc:374] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:57.570540 22496 raft_consensus.cc:729] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106191e21215457892ec751f5e401f76, State: Initialized, Role: FOLLOWER
I20260926 22:44:57.570591 22496 consensus_queue.cc:260] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.570771 22497 tablet_bootstrap.cc:492] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: No bootstrap required, opened a new log
I20260926 22:44:57.570796 22496 ts_tablet_manager.cc:1428] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:57.570822 22497 ts_tablet_manager.cc:1397] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:57.570997 22497 raft_consensus.cc:348] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.571064 22497 raft_consensus.cc:374] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:57.571077 22497 raft_consensus.cc:729] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d19e92a1a4434b80386a7678cc060f, State: Initialized, Role: FOLLOWER
I20260926 22:44:57.571156 22497 consensus_queue.cc:260] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.571237 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.570980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:33495)
0926 22:44:57.571137 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.571172 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260926 22:44:57.571300 22497 ts_tablet_manager.cc:1428] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:57.571641 22270 heartbeater.cc:503] Master 127.14.184.252:40827 was elected leader, sending a full tablet report...
I20260926 22:44:57.571715 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.571505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:50995)
0926 22:44:57.571646 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.571680 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260926 22:44:57.571735 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.571560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:34549)
0926 22:44:57.571617 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.571710 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 22:44:57.571985 22406 heartbeater.cc:503] Master 127.14.184.252:40827 was elected leader, sending a full tablet report...
I20260926 22:44:57.572314 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.571913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:40173)
0926 22:44:57.572258 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.572287 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331}
I20260926 22:44:57.638849 22500 raft_consensus.cc:482] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:57.638949 22500 raft_consensus.cc:504] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.639463 22500 leader_election.cc:288] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 106191e21215457892ec751f5e401f76 (127.14.184.194:39213), a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289)
I20260926 22:44:57.642684 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.639741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:58095 (local address 127.14.184.195:45289)
0926 22:44:57.639792 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.639794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.639803 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:57.639831 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.639832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.639848 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.639902 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.640081 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.640870 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.641956 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.642310 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.642406 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.642416 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.642543 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.642601 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:57.642706 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.639680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:46855 (local address 127.14.184.194:39213)
0926 22:44:57.639726 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.639728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.639950 (+ 222us) server_negotiation.cc:407] Connection header received
0926 22:44:57.639981 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.639982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.640014 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.640084 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:57.640240 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.640932 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.641942 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.642307 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.642398 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.642409 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.642578 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.642617 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:57.642724 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.639611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:39213 (local address 127.14.184.193:46855)
0926 22:44:57.639849 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.639857 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.639935 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.640132 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.640133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.640134 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.640168 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.640169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.640995 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.640996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.641841 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.641843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.642405 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.642406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.642491 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.642495 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.642557 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.642599 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":88,"thread_start_us":115,"threads_started":1}
I20260926 22:44:57.643040 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.639611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:45289 (local address 127.14.184.193:58095)
0926 22:44:57.639659 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.639672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:57.639726 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:57.639935 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:57.639937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:57.639939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:57.639999 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.640001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.640974 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.640975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.641844 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:57.641847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.642398 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.642399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:57.642466 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.642469 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:57.642537 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 22:44:57.642579 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":49}
I20260926 22:44:57.643080 22224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c94a8e3b389044f9a689f75bdad142b8" candidate_uuid: "1d02d950a5e04659a5cba4233efd171f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106191e21215457892ec751f5e401f76" is_pre_election: true
I20260926 22:44:57.643185 22224 raft_consensus.cc:2454] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d02d950a5e04659a5cba4233efd171f in term 0.
I20260926 22:44:57.643296 22360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c94a8e3b389044f9a689f75bdad142b8" candidate_uuid: "1d02d950a5e04659a5cba4233efd171f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d19e92a1a4434b80386a7678cc060f" is_pre_election: true
I20260926 22:44:57.643427 22360 raft_consensus.cc:2454] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d02d950a5e04659a5cba4233efd171f in term 0.
I20260926 22:44:57.643636 22022 leader_election.cc:302] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106191e21215457892ec751f5e401f76, 1d02d950a5e04659a5cba4233efd171f; no voters:
I20260926 22:44:57.643796 22500 raft_consensus.cc:2781] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:57.643842 22500 raft_consensus.cc:482] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:57.643873 22500 raft_consensus.cc:3037] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:57.644557 22500 raft_consensus.cc:504] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.644711 22500 leader_election.cc:288] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [CANDIDATE]: Term 1 election: Requested vote from peers 106191e21215457892ec751f5e401f76 (127.14.184.194:39213), a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289)
I20260926 22:44:57.644889 22360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c94a8e3b389044f9a689f75bdad142b8" candidate_uuid: "1d02d950a5e04659a5cba4233efd171f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d19e92a1a4434b80386a7678cc060f"
I20260926 22:44:57.644964 22360 raft_consensus.cc:3037] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:57.645587 22360 raft_consensus.cc:2454] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d02d950a5e04659a5cba4233efd171f in term 1.
I20260926 22:44:57.645715 22224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c94a8e3b389044f9a689f75bdad142b8" candidate_uuid: "1d02d950a5e04659a5cba4233efd171f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106191e21215457892ec751f5e401f76"
I20260926 22:44:57.645759 22020 leader_election.cc:302] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d02d950a5e04659a5cba4233efd171f, a5d19e92a1a4434b80386a7678cc060f; no voters:
I20260926 22:44:57.645789 22224 raft_consensus.cc:3037] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:57.645874 22500 raft_consensus.cc:2781] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:57.646041 22500 raft_consensus.cc:686] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [term 1 LEADER]: Becoming Leader. State: Replica: 1d02d950a5e04659a5cba4233efd171f, State: Running, Role: LEADER
I20260926 22:44:57.646113 22500 consensus_queue.cc:237] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } }
I20260926 22:44:57.646608 22224 raft_consensus.cc:2454] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d02d950a5e04659a5cba4233efd171f in term 1.
I20260926 22:44:57.646831 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.646736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:58623)
0926 22:44:57.646774 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.646812 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:44:57.647018 21953 catalog_manager.cc:5495] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d02d950a5e04659a5cba4233efd171f (127.14.184.193). New cstate: current_term: 1 leader_uuid: "1d02d950a5e04659a5cba4233efd171f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d02d950a5e04659a5cba4233efd171f" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 41781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:57.657671 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.655730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.14.184.193:41781)
0926 22:44:57.655767 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:57.655769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:57.655784 (+ 15us) server_negotiation.cc:407] Connection header received
0926 22:44:57.655828 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:57.655828 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:57.655842 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:57.655891 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:57.655976 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.656414 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.657093 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:57.657261 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:57.657307 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:57.657436 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:57.657524 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:57.657576 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:57.657612 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 22:44:57.657629 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:44:57.658618 22224 raft_consensus.cc:1261] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Refusing update from remote peer 1d02d950a5e04659a5cba4233efd171f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:57.658715 22360 raft_consensus.cc:1261] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Refusing update from remote peer 1d02d950a5e04659a5cba4233efd171f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:57.658885 22508 consensus_queue.cc:1035] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [LEADER]: Connected to new peer: Peer: permanent_uuid: "106191e21215457892ec751f5e401f76" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:57.658998 22500 consensus_queue.cc:1035] T c94a8e3b389044f9a689f75bdad142b8 P 1d02d950a5e04659a5cba4233efd171f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d19e92a1a4434b80386a7678cc060f" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:57.659782 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.659553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:46281)
0926 22:44:57.659717 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.659751 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260926 22:44:57.659811 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.659611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:52439)
0926 22:44:57.659757 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.659781 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260926 22:44:57.660367 22511 mvcc.cc:205] Tried to move back new op lower bound from 7333735209607925760 to 7333735209559080960. Current Snapshot: MvccSnapshot[applied={T|T < 7333735209607925760}]
I20260926 22:44:57.660908 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.660787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:41473)
0926 22:44:57.660837 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.660877 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 22:44:57.661206 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.661122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.195:42335)
0926 22:44:57.661153 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.661178 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 22:44:57.661621 22501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.661540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.193:36427)
0926 22:44:57.661570 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.661598 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260926 22:44:57.661985 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.661888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43809 (local address 127.14.184.194:48297)
0926 22:44:57.661941 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.661962 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 22:44:57.664398 21953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:57.664489 21953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:57.665705 21953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f088109a197045c68e1552445490331b: Sending DeleteTablet for 3 replicas of tablet c94a8e3b389044f9a689f75bdad142b8
I20260926 22:44:57.666018 22067 tablet_service.cc:1514] Processing DeleteTablet for tablet c94a8e3b389044f9a689f75bdad142b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:57 UTC) from {username='oryx', principal='oryx/127.14.184.252@KRBTEST.COM'} at 127.0.0.1:49034
I20260926 22:44:57.666044 22340 tablet_service.cc:1514] Processing DeleteTablet for tablet c94a8e3b389044f9a689f75bdad142b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:57 UTC) from {username='oryx', principal='oryx/127.14.184.252@KRBTEST.COM'} at 127.0.0.1:46300
I20260926 22:44:57.666188 22204 tablet_service.cc:1514] Processing DeleteTablet for tablet c94a8e3b389044f9a689f75bdad142b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:57 UTC) from {username='oryx', principal='oryx/127.14.184.252@KRBTEST.COM'} at 127.0.0.1:58332
I20260926 22:44:57.666210 22522 tablet_replica.cc:330] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: stopping tablet replica
I20260926 22:44:57.666260 22522 raft_consensus.cc:2229] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:57.666313 22522 raft_consensus.cc:2258] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:57.666579 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21916
I20260926 22:44:57.667236 22522 ts_tablet_manager.cc:1905] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:57.668716 22522 ts_tablet_manager.cc:1918] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:57.668764 22522 log.cc:1196] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/c94a8e3b389044f9a689f75bdad142b8
I20260926 22:44:57.669014 22522 ts_tablet_manager.cc:1939] T c94a8e3b389044f9a689f75bdad142b8 P a5d19e92a1a4434b80386a7678cc060f: Deleting consensus metadata
I20260926 22:44:57.669312 21938 catalog_manager.cc:4841] TS a5d19e92a1a4434b80386a7678cc060f (127.14.184.195:45289): tablet c94a8e3b389044f9a689f75bdad142b8 (table test-table [id=1922d66fa8c946849e0183e5e9389a11]) successfully deleted
I20260926 22:44:57.672438 22523 tablet_replica.cc:330] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76: stopping tablet replica
I20260926 22:44:57.672504 22523 raft_consensus.cc:2229] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:57.672564 22523 raft_consensus.cc:2258] T c94a8e3b389044f9a689f75bdad142b8 P 106191e21215457892ec751f5e401f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:57.672590 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 22141
I20260926 22:44:57.672706 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.672598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:41781 (local address 127.0.0.1:49046)
0926 22:44:57.672637 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.672669 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:41781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260926 22:44:57.673463 21938 catalog_manager.cc:4584] TS 1d02d950a5e04659a5cba4233efd171f (127.14.184.193:41781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c94a8e3b389044f9a689f75bdad142b8: Network error: Client connection negotiation failed: client connection to 127.14.184.193:41781: connect: Connection refused (error 111)
I20260926 22:44:57.677526 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:57.677411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:39213 (local address 127.0.0.1:58338)
0926 22:44:57.677460 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:57.677485 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.194:39213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 22:44:57.677582 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 22277
I20260926 22:44:57.683014 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21634
I20260926 22:44:57.687637 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 21823
2026-09-26T22:44:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4702 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:44:57 dist-test-slave-3986 krb5kdc[22526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:44:57 dist-test-slave-3986 krb5kdc[22526](info): set up 2 sockets
Sep 26 22:44:57 dist-test-slave-3986 krb5kdc[22526](info): commencing operation
krb5kdc: starting...
W20260926 22:44:58.715447 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:44:58 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:44:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab.
Entry for principal oryx/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab.
I20260926 22:44:58.829526 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45623
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab
--principal=oryx/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:45623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:58.908499 22539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:58.908701 22539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:58.908722 22539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:58.909817 22539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:44:58.909857 22539 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 22:44:58.909869 22539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:58.909883 22539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:58.909894 22539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:44:58.909931 22539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:44:58.911393 22539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:45623
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.254.keytab
--principal=oryx/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:58.911633 22539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:58.911850 22539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:58.916785 22545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:58.916817 22544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:58.916791 22547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:58.916940 22539 server_base.cc:1034] running on GCE node
I20260926 22:44:58.917271 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:58.917604 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:58.918754 22539 hybrid_clock.cc:648] HybridClock initialized: now 1790462698918736 us; error 46 us; skew 500 ppm
Sep 26 22:44:58 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462698, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:58.921167 22539 init.cc:377] Logged in from keytab as oryx/127.14.184.254@KRBTEST.COM (short username oryx)
I20260926 22:44:58.921638 22539 webserver.cc:460] Webserver started at http://127.14.184.254:36857/ using document root <none> and password file <none>
I20260926 22:44:58.921872 22539 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:58.921938 22539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:58.922071 22539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:58.923177 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "775214a523514a0289b14f62e255e87b"
format_stamp: "Formatted at 2026-09-26 22:44:58 on dist-test-slave-3986"
server_key: "9ff8bfd72595e92ead1dfa1cdecc33b2"
server_key_iv: "a7dd8f579a88f181c74607d7087bf78a"
server_key_version: "encryptionkey@0"
I20260926 22:44:58.923592 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "775214a523514a0289b14f62e255e87b"
format_stamp: "Formatted at 2026-09-26 22:44:58 on dist-test-slave-3986"
server_key: "9ff8bfd72595e92ead1dfa1cdecc33b2"
server_key_iv: "a7dd8f579a88f181c74607d7087bf78a"
server_key_version: "encryptionkey@0"
I20260926 22:44:58.924873 22539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:44:58.925617 22554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:58.925806 22539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:58.925904 22539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "775214a523514a0289b14f62e255e87b"
format_stamp: "Formatted at 2026-09-26 22:44:58 on dist-test-slave-3986"
server_key: "9ff8bfd72595e92ead1dfa1cdecc33b2"
server_key_iv: "a7dd8f579a88f181c74607d7087bf78a"
server_key_version: "encryptionkey@0"
I20260926 22:44:58.925987 22539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:58.936980 22539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:58.938736 22539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:58.938884 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:58.943557 22539 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:45623
I20260926 22:44:58.943598 22616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:45623 every 8 connection(s)
I20260926 22:44:58.943923 22539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:44:58.944519 22617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:58.946612 22617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Bootstrap starting.
I20260926 22:44:58.947188 22617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:58.947422 22617 log.cc:824] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:58.948071 22617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: No bootstrap required, opened a new log
I20260926 22:44:58.949998 22617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775214a523514a0289b14f62e255e87b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45623 } }
I20260926 22:44:58.950124 22617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:58.950145 22617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775214a523514a0289b14f62e255e87b, State: Initialized, Role: FOLLOWER
I20260926 22:44:58.950240 22617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775214a523514a0289b14f62e255e87b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45623 } }
I20260926 22:44:58.950322 22617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:44:58.950366 22617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:44:58.950426 22617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:58.951010 22617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775214a523514a0289b14f62e255e87b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45623 } }
I20260926 22:44:58.951128 22617 leader_election.cc:302] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 775214a523514a0289b14f62e255e87b; no voters:
I20260926 22:44:58.951303 22617 leader_election.cc:288] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:44:58.951378 22620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:58.951534 22620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [term 1 LEADER]: Becoming Leader. State: Replica: 775214a523514a0289b14f62e255e87b, State: Running, Role: LEADER
I20260926 22:44:58.951599 22617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:44:58.951625 22620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775214a523514a0289b14f62e255e87b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45623 } }
I20260926 22:44:58.952014 22621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "775214a523514a0289b14f62e255e87b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775214a523514a0289b14f62e255e87b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45623 } } }
I20260926 22:44:58.952044 22622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 775214a523514a0289b14f62e255e87b. Latest consensus state: current_term: 1 leader_uuid: "775214a523514a0289b14f62e255e87b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775214a523514a0289b14f62e255e87b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45623 } } }
I20260926 22:44:58.952112 22621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:58.952116 22622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b [sys.catalog]: This master's current role is: LEADER
I20260926 22:44:58.952435 22629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:44:58.952920 22629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:44:58.953956 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 22539
I20260926 22:44:58.954079 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:44:58.954291 15075 external_mini_cluster.cc:1428] Setting key b5d295fd0fbfc3048737d036f4e61998
I20260926 22:44:58.954484 22629 catalog_manager.cc:1347] Generated new cluster ID: 892a3fb7e0a74c7789cefc40066391b0
I20260926 22:44:58.954545 22629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:44:58 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:58.965857 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:58.954915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.14.184.254:45623)
0926 22:44:58.955062 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:58.955071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:58.955101 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:44:58.955206 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:58.955213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:58.955264 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:58.955380 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:58.956293 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:58.957628 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:58.958811 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:58.959402 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:58.959499 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:58.962274 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:58.962278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:58.962279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:58.962291 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:58.964270 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:58.964637 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:58.964640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:58.964641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:58.964761 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:58.964967 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:58.964968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:58.964968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:58.965258 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:58.965411 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:58.965481 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 22:44:58.965615 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260926 22:44:58.970541 22629 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:44:58.971043 22629 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:44:58.975199 22629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Generated new TSK 0
I20260926 22:44:58.975371 22629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab.
Entry for principal oryx/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab.
I20260926 22:44:59.005203 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45623
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:59.084023 22643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:59.084219 22643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:59.084242 22643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:59.085400 22643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:59.085439 22643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:59.085459 22643 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:59.085520 22643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:44:59.086994 22643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:59.087237 22643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:59.087471 22643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:59.092463 22649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:59.092454 22651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:59.092456 22648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:59.092679 22643 server_base.cc:1034] running on GCE node
I20260926 22:44:59.092876 22643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:59.093060 22643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:59.094233 22643 hybrid_clock.cc:648] HybridClock initialized: now 1790462699094221 us; error 37 us; skew 500 ppm
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:59.096822 22643 init.cc:377] Logged in from keytab as oryx/127.14.184.193@KRBTEST.COM (short username oryx)
I20260926 22:44:59.097183 22643 webserver.cc:460] Webserver started at http://127.14.184.193:41883/ using document root <none> and password file <none>
I20260926 22:44:59.097352 22643 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:59.097398 22643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:59.097484 22643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:59.098425 22643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "4b055ce614ab0b4f895e30c3ef2d841b"
server_key_iv: "10a012b7f7e802e49b9e66da4dd246ac"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.098791 22643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "4b055ce614ab0b4f895e30c3ef2d841b"
server_key_iv: "10a012b7f7e802e49b9e66da4dd246ac"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.100013 22643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:44:59.100642 22658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.100806 22643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:59.100886 22643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "4b055ce614ab0b4f895e30c3ef2d841b"
server_key_iv: "10a012b7f7e802e49b9e66da4dd246ac"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.100963 22643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:59.113850 22643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:59.115697 22643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:59.115846 22643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:59.116321 22643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:59.116358 22643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.116379 22643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:59.116392 22643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.122884 22643 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:46289
I20260926 22:44:59.122956 22773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:46289 every 8 connection(s)
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:59.123281 22643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:44:59.126200 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.116798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40713 (local address 127.14.184.254:45623)
0926 22:44:59.116874 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.116877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.117029 (+ 152us) server_negotiation.cc:407] Connection header received
0926 22:44:59.117505 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.117507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.117523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.117601 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.118484 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.119340 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.120558 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.120900 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.120977 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.123578 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.123580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.123580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.123591 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.125028 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.125410 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.125410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.125411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.125460 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.125662 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.125662 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.125663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.125755 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.125806 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.126069 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.126137 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 22:44:59.126392 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.116755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.14.184.193:40713)
0926 22:44:59.116901 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.116939 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.117408 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.117697 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.117701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.118092 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.118373 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.118380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.119457 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.119458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.120480 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.120484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.120995 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.120996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.121139 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.121696 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.121708 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.123478 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.125110 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.125117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.125118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.125336 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.125516 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.125516 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.125517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.125610 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.125830 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.125836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.125955 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.126045 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.126161 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260926 22:44:59.126958 22774 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45623
I20260926 22:44:59.127043 22774 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:59.127211 22774 heartbeater.cc:511] Master 127.14.184.254:45623 requested a full tablet report, sending...
I20260926 22:44:59.127748 22570 ts_manager.cc:194] Registered new tserver with Master: a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289)
I20260926 22:44:59.128633 22570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:40713
I20260926 22:44:59.130343 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 22643
I20260926 22:44:59.130477 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:44:59.130738 15075 external_mini_cluster.cc:1428] Setting key 612f76cc3e812165a3741ae9c507ae31
I20260926 22:44:59.133638 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.128015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:32773 (local address 127.14.184.254:45623)
0926 22:44:59.128049 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.128051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.128112 (+ 61us) server_negotiation.cc:407] Connection header received
0926 22:44:59.128176 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.128177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.128186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.128236 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.128550 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.129277 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.129982 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.130192 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.130269 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.131210 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.131211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.131211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.131217 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.132531 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.132882 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.132883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.132883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.132948 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.133144 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.133145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.133145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.133252 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.133325 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.133477 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.133549 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:59.133647 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.127939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.14.184.193:32773)
0926 22:44:59.128056 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.128066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.128122 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.128259 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.128260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.128408 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.128459 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.128460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.129362 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.129363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.129912 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.129913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.130265 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.130265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.130310 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.130820 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.130825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.131134 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.132629 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.132630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.132630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.132797 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.133014 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.133014 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.133015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.133080 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.133318 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.133319 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.133362 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.133452 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.133551 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for oryx/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab.
Entry for principal oryx/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab.
I20260926 22:44:59.155026 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45623
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:59.234009 22783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:59.234202 22783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:59.234225 22783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:59.235312 22783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:59.235347 22783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:59.235366 22783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:59.235383 22783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:44:59.236786 22783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:59.236967 22783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:59.237131 22783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:44:59.241899 22783 server_base.cc:1034] running on GCE node
W20260926 22:44:59.241937 22791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:59.241986 22789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:59.242216 22788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:59.242494 22783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:59.242735 22783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:59.243882 22783 hybrid_clock.cc:648] HybridClock initialized: now 1790462699243865 us; error 40 us; skew 500 ppm
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:59.246197 22783 init.cc:377] Logged in from keytab as oryx/127.14.184.194@KRBTEST.COM (short username oryx)
I20260926 22:44:59.246610 22783 webserver.cc:460] Webserver started at http://127.14.184.194:35701/ using document root <none> and password file <none>
I20260926 22:44:59.246851 22783 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:59.246917 22783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:59.247044 22783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:59.248035 22783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "dee8e9d391e248a8b9df535e47c4a49f"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "a1822dfb0430ea87d183822e10e3ea49"
server_key_iv: "b1e4dddc30ece7d06a923ff46b245747"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.248375 22783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "dee8e9d391e248a8b9df535e47c4a49f"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "a1822dfb0430ea87d183822e10e3ea49"
server_key_iv: "b1e4dddc30ece7d06a923ff46b245747"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.249648 22783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:44:59.250327 22798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.250507 22783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:44:59.250602 22783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "dee8e9d391e248a8b9df535e47c4a49f"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "a1822dfb0430ea87d183822e10e3ea49"
server_key_iv: "b1e4dddc30ece7d06a923ff46b245747"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.250694 22783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:59.276512 22783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:59.278290 22783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:59.278455 22783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:59.279022 22783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:59.279086 22783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.279129 22783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:59.279156 22783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:59.286219 22783 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:34427
I20260926 22:44:59.286316 22913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:34427 every 8 connection(s)
I20260926 22:44:59.286602 22783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:44:59.288487 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.279499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:49865 (local address 127.14.184.254:45623)
0926 22:44:59.279544 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.279546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.279742 (+ 196us) server_negotiation.cc:407] Connection header received
0926 22:44:59.280254 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.280255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.280272 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.280342 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.281220 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.282050 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.283304 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.283565 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.283637 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.285755 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.285757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.285758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.285766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.287391 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.287742 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.287743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.287743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.287794 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.287999 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.288000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.288000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.288097 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.288147 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.288365 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.288426 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:44:59.288712 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.279489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.14.184.194:49865)
0926 22:44:59.279633 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.279668 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.280149 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.280436 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.280440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.280808 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.281106 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.281112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.282145 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.282146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.283207 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.283210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.283658 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.283659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.283782 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.284290 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.284298 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.285655 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.287464 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.287468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.287469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.287675 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.287850 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.287851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.287851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.287946 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.288152 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.288156 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.288260 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.288340 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.288460 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260926 22:44:59.289162 22914 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45623
I20260926 22:44:59.289255 22914 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:59.289466 22914 heartbeater.cc:511] Master 127.14.184.254:45623 requested a full tablet report, sending...
I20260926 22:44:59.289580 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 22783
I20260926 22:44:59.289695 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:44:59.289856 22571 ts_manager.cc:194] Registered new tserver with Master: dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427)
I20260926 22:44:59.289906 15075 external_mini_cluster.cc:1428] Setting key 8ba807d12e1ac0adfba9a8043ac9c063
I20260926 22:44:59.290342 22571 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:49865
I20260926 22:44:59.295708 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.290255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:52873 (local address 127.14.184.254:45623)
0926 22:44:59.290291 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.290293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.290355 (+ 62us) server_negotiation.cc:407] Connection header received
0926 22:44:59.290453 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.290454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.290464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.290519 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.291053 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.291509 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.292195 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.292403 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.292486 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.293283 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.293285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.293285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.293292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.294526 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.294910 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.294911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.294912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.294966 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.295177 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.295178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.295179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.295318 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.295395 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.295550 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.295623 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:59.295727 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.290163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.14.184.194:52873)
0926 22:44:59.290299 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.290310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.290378 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.290659 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.290660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.290906 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.290976 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.290977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.291602 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.291603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.292113 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.292114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.292488 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.292489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.292555 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.292948 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.292952 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.293196 (+ 244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.294618 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.294619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.294619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.294832 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.295040 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.295040 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.295041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.295098 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.295390 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.295390 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.295437 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.295525 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.295624 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab.
Entry for principal oryx/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab.
I20260926 22:44:59.314994 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45623
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:44:59.394881 22923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:44:59.395112 22923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:44:59.395148 22923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:44:59.396291 22923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:44:59.396354 22923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:44:59.396385 22923 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 22:44:59.396414 22923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:44:59.397943 22923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.184.254:45623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:44:59.398208 22923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:44:59.398434 22923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:44:59.403389 22929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:59.403487 22923 server_base.cc:1034] running on GCE node
W20260926 22:44:59.403393 22928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:44:59.403395 22931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:44:59.403848 22923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:44:59.404111 22923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:44:59.405277 22923 hybrid_clock.cc:648] HybridClock initialized: now 1790462699405283 us; error 49 us; skew 500 ppm
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:44:59.407716 22923 init.cc:377] Logged in from keytab as oryx/127.14.184.195@KRBTEST.COM (short username oryx)
I20260926 22:44:59.408145 22923 webserver.cc:460] Webserver started at http://127.14.184.195:33015/ using document root <none> and password file <none>
I20260926 22:44:59.408329 22923 fs_manager.cc:359] Metadata directory not provided
I20260926 22:44:59.408368 22923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:44:59.408459 22923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:44:59.409474 22923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "9c80b98fd6614d978b25167f9104f3ef"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "ac01022076fd382bc147144995cdf850"
server_key_iv: "0b4b4ea4f423443a25541e5882108517"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.409842 22923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "9c80b98fd6614d978b25167f9104f3ef"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "ac01022076fd382bc147144995cdf850"
server_key_iv: "0b4b4ea4f423443a25541e5882108517"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.411248 22923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:44:59.411900 22938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.412083 22923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:59.412165 22923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "9c80b98fd6614d978b25167f9104f3ef"
format_stamp: "Formatted at 2026-09-26 22:44:59 on dist-test-slave-3986"
server_key: "ac01022076fd382bc147144995cdf850"
server_key_iv: "0b4b4ea4f423443a25541e5882108517"
server_key_version: "encryptionkey@0"
I20260926 22:44:59.412215 22923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:44:59.423719 22923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:44:59.425412 22923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:44:59.425562 22923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:44:59.426041 22923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:44:59.426082 22923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.426103 22923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:44:59.426144 22923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.433121 22923 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:41031
I20260926 22:44:59.433190 23053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:41031 every 8 connection(s)
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:59.433526 22923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:44:59.436313 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.426547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:36963 (local address 127.14.184.254:45623)
0926 22:44:59.426620 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.426623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.426832 (+ 209us) server_negotiation.cc:407] Connection header received
0926 22:44:59.427409 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.427410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.427426 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.427514 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.428352 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.429203 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.430811 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.431162 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.431244 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.433939 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.433940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.433941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.433948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.435199 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.435564 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.435565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.435565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.435617 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.435787 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.435788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.435788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.435895 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.435947 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.436170 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.436248 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 22:44:59.436473 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.426500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.14.184.195:36963)
0926 22:44:59.426646 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.426741 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.427301 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.427576 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.427579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.427969 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.428250 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.428256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.429313 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.429314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.430721 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.430725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.431258 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.431258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.431395 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.431944 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.431954 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.433862 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.435274 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.435280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.435281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.435498 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.435658 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.435659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.435659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.435739 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.435947 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.435950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.436056 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.436139 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.436257 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260926 22:44:59.436985 23054 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45623
I20260926 22:44:59.437072 23054 heartbeater.cc:464] Registering TS with master...
I20260926 22:44:59.437251 23054 heartbeater.cc:511] Master 127.14.184.254:45623 requested a full tablet report, sending...
I20260926 22:44:59.437644 22570 ts_manager.cc:194] Registered new tserver with Master: 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:44:59.438113 22570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:36963
I20260926 22:44:59.439111 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 22923
I20260926 22:44:59.439234 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:44:59.439452 15075 external_mini_cluster.cc:1428] Setting key 862b280a5cd71201eb6d3e63bfe7d27a
I20260926 22:44:59.440331 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:44:59.444121 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.438016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41385 (local address 127.14.184.254:45623)
0926 22:44:59.438058 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.438060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.438179 (+ 119us) server_negotiation.cc:407] Connection header received
0926 22:44:59.438279 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.438280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.438292 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.438355 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.438741 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.439262 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.440075 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.440358 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.440426 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.441368 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.441369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.441369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.441376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.443037 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.443373 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.443375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.443375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.443433 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.443623 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.443624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.443624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.443731 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.443808 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.443957 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.444031 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:59.444139 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.437958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.14.184.195:41385)
0926 22:44:59.438110 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.438120 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.438193 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.438360 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.438360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.438564 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.438620 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.438622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.439342 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.439344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.440017 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.440019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.440440 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.440441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.440505 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.440881 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.440886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.441283 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.443120 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.443121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.443121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.443280 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.443500 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.443501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.443502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.443561 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.443799 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.443800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.443845 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.443931 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.444033 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 22:44:59.461721 22640 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:59.456709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.14.184.254:45623)
0926 22:44:59.456775 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.456776 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.456784 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:59.456819 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.456819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.456827 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.456875 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.457393 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.457954 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.458637 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.458852 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.458908 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.461624 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40696: BlockingRecv error: recv got EOF from 127.0.0.1:40696 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:59.468974 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.463109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.14.184.254:45623)
0926 22:44:59.463165 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.463166 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.463215 (+ 49us) server_negotiation.cc:407] Connection header received
0926 22:44:59.463263 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.463264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.463271 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.463322 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.463707 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.464135 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.464742 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.464912 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.464958 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.466937 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.466938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.466938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.466943 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.468129 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.468423 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.468424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.468424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.468472 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.468627 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.468627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.468627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.468715 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.468785 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.468860 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.468918 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:44:59.470320 22570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:59.471817 22570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:59.478075 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.474032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.14.184.193:46289)
0926 22:44:59.474170 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.474176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.474482 (+ 306us) server_negotiation.cc:407] Connection header received
0926 22:44:59.474527 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.474529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.474544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.474613 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.474854 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.475527 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477162 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.477651 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477762 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.477822 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.477930 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.477972 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260926 22:44:59.478101 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.473789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:41031 (local address 127.0.0.1:39548)
0926 22:44:59.474200 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.474209 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.474261 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.474531 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.474532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.474535 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.474574 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.474575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.475806 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.475807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.476841 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.476843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.477829 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.477891 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.477895 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.477955 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.477995 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":75,"thread_start_us":48,"threads_started":1}
I20260926 22:44:59.478084 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.473798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:46289 (local address 127.0.0.1:59802)
0926 22:44:59.474344 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.474353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.474423 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.474725 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.474726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.474728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.474775 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.474777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.475769 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.475770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.477097 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.477098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.477742 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.477819 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.477833 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.477914 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.477969 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":44,"threads_started":1}
I20260926 22:44:59.478263 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.473958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.14.184.194:34427)
0926 22:44:59.474135 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.474140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.474151 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:44:59.474242 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.474244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.474261 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.474341 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.474513 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.475671 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477089 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.477781 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477860 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.477907 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.478126 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.478172 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260926 22:44:59.478350 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.473943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39548 (local address 127.14.184.195:41031)
0926 22:44:59.474222 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.474225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.474258 (+ 33us) server_negotiation.cc:407] Connection header received
0926 22:44:59.474354 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.474357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.474372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.474465 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.474649 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.475726 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.476928 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.477646 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477735 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.477792 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.478217 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.478258 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":78,"threads_started":1}
I20260926 22:44:59.478363 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.473786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:34427 (local address 127.0.0.1:38300)
0926 22:44:59.474026 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.474066 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.474133 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.474357 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.474360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.474362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.474427 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.474429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.475755 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.475756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.477010 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.477013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.477938 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.477939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.478006 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.478009 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.478055 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.478089 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":67,"thread_start_us":43,"threads_started":1}
I20260926 22:44:59.479321 22848 tablet_service.cc:1467] Processing CreateTablet for tablet 9a6f50c9cf8943ccb6036592df5f7a0b (DEFAULT_TABLE table=test-table [id=d8215a41faf74a4bb7e1ed845ffd232a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.479423 22708 tablet_service.cc:1467] Processing CreateTablet for tablet 9a6f50c9cf8943ccb6036592df5f7a0b (DEFAULT_TABLE table=test-table [id=d8215a41faf74a4bb7e1ed845ffd232a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.479707 22848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a6f50c9cf8943ccb6036592df5f7a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.479720 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a6f50c9cf8943ccb6036592df5f7a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.483037 22981 tablet_service.cc:1467] Processing CreateTablet for tablet 9a6f50c9cf8943ccb6036592df5f7a0b (DEFAULT_TABLE table=test-table [id=d8215a41faf74a4bb7e1ed845ffd232a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.483337 22981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a6f50c9cf8943ccb6036592df5f7a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.483614 23079 tablet_bootstrap.cc:492] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Bootstrap starting.
I20260926 22:44:59.483809 23080 tablet_bootstrap.cc:492] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Bootstrap starting.
I20260926 22:44:59.484426 23080 tablet_bootstrap.cc:654] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.484759 23080 log.cc:824] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:59.485486 23080 tablet_bootstrap.cc:492] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: No bootstrap required, opened a new log
I20260926 22:44:59.485570 23080 ts_tablet_manager.cc:1397] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:59.485571 23079 tablet_bootstrap.cc:654] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.485865 23079 log.cc:824] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:59.486646 23079 tablet_bootstrap.cc:492] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: No bootstrap required, opened a new log
I20260926 22:44:59.486735 23079 ts_tablet_manager.cc:1397] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 22:44:59.487174 23083 tablet_bootstrap.cc:492] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Bootstrap starting.
I20260926 22:44:59.487365 23080 raft_consensus.cc:348] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.487560 23080 raft_consensus.cc:374] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.487603 23080 raft_consensus.cc:729] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.487723 23080 consensus_queue.cc:260] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.487933 23083 tablet_bootstrap.cc:654] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.487987 23080 ts_tablet_manager.cc:1428] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 22:44:59.488111 22914 heartbeater.cc:503] Master 127.14.184.254:45623 was elected leader, sending a full tablet report...
I20260926 22:44:59.488308 23083 log.cc:824] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Log is configured to *not* fsync() on all Append() calls
I20260926 22:44:59.488641 23079 raft_consensus.cc:348] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.488796 23079 raft_consensus.cc:374] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.488842 23079 raft_consensus.cc:729] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bb79a203224c0a9eddcc5c80d9c5b7, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.488997 23083 tablet_bootstrap.cc:492] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: No bootstrap required, opened a new log
I20260926 22:44:59.488984 23079 consensus_queue.cc:260] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.489064 23083 ts_tablet_manager.cc:1397] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:59.489234 22774 heartbeater.cc:503] Master 127.14.184.254:45623 was elected leader, sending a full tablet report...
I20260926 22:44:59.489224 23079 ts_tablet_manager.cc:1428] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:44:59.490947 23083 raft_consensus.cc:348] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.491084 23083 raft_consensus.cc:374] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.491127 23083 raft_consensus.cc:729] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c80b98fd6614d978b25167f9104f3ef, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.491235 23083 consensus_queue.cc:260] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.491464 23083 ts_tablet_manager.cc:1428] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 22:44:59.491508 23054 heartbeater.cc:503] Master 127.14.184.254:45623 was elected leader, sending a full tablet report...
I20260926 22:44:59.520875 23087 raft_consensus.cc:482] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:59.521016 23087 raft_consensus.cc:504] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.521364 23087 leader_election.cc:288] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427), a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289)
I20260926 22:44:59.525182 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.521675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41643 (local address 127.14.184.194:34427)
0926 22:44:59.521719 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.521722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.521730 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:59.521756 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.521757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.521765 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.521822 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.522006 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.522805 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.524268 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.524802 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.524877 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.524885 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.525071 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.525113 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:44:59.525187 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.521654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:35785 (local address 127.14.184.193:46289)
0926 22:44:59.521700 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.521702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.521892 (+ 190us) server_negotiation.cc:407] Connection header received
0926 22:44:59.521929 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.521929 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.521937 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.522000 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.522141 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.522845 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.524265 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.524802 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.524877 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.524885 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.525076 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.525112 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:44:59.525210 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.521555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:46289 (local address 127.14.184.195:35785)
0926 22:44:59.521779 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.521787 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.521855 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.522003 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.522004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.522005 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.522077 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.522078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.522909 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.522910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.524177 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.524180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.524891 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.524892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.524982 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.524986 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.525046 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.525095 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":98,"thread_start_us":105,"threads_started":1}
I20260926 22:44:59.525221 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.521555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:34427 (local address 127.14.184.195:41643)
0926 22:44:59.521616 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.521630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.521682 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.521841 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.521842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.521843 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.521932 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.521934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.522890 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.522890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.524177 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.524180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.524894 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.524895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.524984 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.524986 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.525046 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.525096 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":41}
I20260926 22:44:59.525804 22868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a6f50c9cf8943ccb6036592df5f7a0b" candidate_uuid: "9c80b98fd6614d978b25167f9104f3ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dee8e9d391e248a8b9df535e47c4a49f" is_pre_election: true
I20260926 22:44:59.525928 22868 raft_consensus.cc:2454] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c80b98fd6614d978b25167f9104f3ef in term 0.
I20260926 22:44:59.526122 22941 leader_election.cc:302] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c80b98fd6614d978b25167f9104f3ef, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.526121 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a6f50c9cf8943ccb6036592df5f7a0b" candidate_uuid: "9c80b98fd6614d978b25167f9104f3ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" is_pre_election: true
I20260926 22:44:59.526273 22728 raft_consensus.cc:2454] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c80b98fd6614d978b25167f9104f3ef in term 0.
I20260926 22:44:59.526330 23087 raft_consensus.cc:2781] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:59.526407 23087 raft_consensus.cc:482] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:59.526446 23087 raft_consensus.cc:3037] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.527464 23087 raft_consensus.cc:504] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.527608 23087 leader_election.cc:288] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [CANDIDATE]: Term 1 election: Requested vote from peers dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427), a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289)
I20260926 22:44:59.527751 22868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a6f50c9cf8943ccb6036592df5f7a0b" candidate_uuid: "9c80b98fd6614d978b25167f9104f3ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dee8e9d391e248a8b9df535e47c4a49f"
I20260926 22:44:59.527773 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a6f50c9cf8943ccb6036592df5f7a0b" candidate_uuid: "9c80b98fd6614d978b25167f9104f3ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
I20260926 22:44:59.527856 22868 raft_consensus.cc:3037] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.527853 22728 raft_consensus.cc:3037] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.528615 22728 raft_consensus.cc:2454] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c80b98fd6614d978b25167f9104f3ef in term 1.
I20260926 22:44:59.528643 22868 raft_consensus.cc:2454] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c80b98fd6614d978b25167f9104f3ef in term 1.
I20260926 22:44:59.528789 22941 leader_election.cc:302] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c80b98fd6614d978b25167f9104f3ef, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.528932 23087 raft_consensus.cc:2781] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:59.529124 23087 raft_consensus.cc:686] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 1 LEADER]: Becoming Leader. State: Replica: 9c80b98fd6614d978b25167f9104f3ef, State: Running, Role: LEADER
I20260926 22:44:59.529238 23087 consensus_queue.cc:237] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } }
I20260926 22:44:59.529981 22570 catalog_manager.cc:5495] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195). New cstate: current_term: 1 leader_uuid: "9c80b98fd6614d978b25167f9104f3ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } health_report { overall_health: UNKNOWN } } }
W20260926 22:44:59.537117 22915 tablet.cc:2369] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:59.549682 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.547474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.14.184.195:41031)
0926 22:44:59.547557 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.547559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.547569 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:59.547607 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.547608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.547625 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.547694 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:59.547859 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.548325 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.549063 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.549245 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.549304 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.549434 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.549524 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.549582 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.549616 (+ 34us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.549632 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 22:44:59.551412 22868 raft_consensus.cc:1261] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Refusing update from remote peer 9c80b98fd6614d978b25167f9104f3ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:59.551471 22728 raft_consensus.cc:1261] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Refusing update from remote peer 9c80b98fd6614d978b25167f9104f3ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:59.551802 23087 consensus_queue.cc:1035] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:59.551954 23089 consensus_queue.cc:1035] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:59.554248 23092 mvcc.cc:205] Tried to move back new op lower bound from 7333735217360240640 to 7333735217272619008. Current Snapshot: MvccSnapshot[applied={T|T < 7333735217360240640}]
I20260926 22:44:59.554896 23093 mvcc.cc:205] Tried to move back new op lower bound from 7333735217360240640 to 7333735217272619008. Current Snapshot: MvccSnapshot[applied={T|T < 7333735217360240640}]
I20260926 22:44:59.557060 22570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:59.557144 22570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:59.557849 22570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Sending DeleteTablet for 3 replicas of tablet 9a6f50c9cf8943ccb6036592df5f7a0b
I20260926 22:44:59.558157 22981 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a6f50c9cf8943ccb6036592df5f7a0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:59 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39548
I20260926 22:44:59.558315 22848 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a6f50c9cf8943ccb6036592df5f7a0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:59 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38300
I20260926 22:44:59.558377 23102 tablet_replica.cc:330] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: stopping tablet replica
I20260926 22:44:59.558346 22708 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a6f50c9cf8943ccb6036592df5f7a0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:59 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:59802
I20260926 22:44:59.558467 23102 raft_consensus.cc:2229] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:59.558583 23102 raft_consensus.cc:2258] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:59.558589 22570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 22:44:59.558717 23104 tablet_replica.cc:330] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: stopping tablet replica
W20260926 22:44:59.558749 22570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:59.558801 23104 raft_consensus.cc:2229] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:59.558900 23104 raft_consensus.cc:2258] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:59.559038 23103 tablet_replica.cc:330] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: stopping tablet replica
I20260926 22:44:59.559122 23103 raft_consensus.cc:2229] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:59.559235 23103 raft_consensus.cc:2258] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:59.559665 23104 ts_tablet_manager.cc:1905] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:59.559794 23102 ts_tablet_manager.cc:1905] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:59.559943 22708 tablet_service.cc:1467] Processing CreateTablet for tablet 74494f0897194eb5b7d4ba7ebd3c215b (DEFAULT_TABLE table=test-table [id=15d139da6ee948e89e23ae54cbbcf041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.559942 22981 tablet_service.cc:1467] Processing CreateTablet for tablet 74494f0897194eb5b7d4ba7ebd3c215b (DEFAULT_TABLE table=test-table [id=15d139da6ee948e89e23ae54cbbcf041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.560047 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74494f0897194eb5b7d4ba7ebd3c215b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.560047 22981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74494f0897194eb5b7d4ba7ebd3c215b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.561298 23083 tablet_bootstrap.cc:492] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Bootstrap starting.
I20260926 22:44:59.561322 23079 tablet_bootstrap.cc:492] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Bootstrap starting.
I20260926 22:44:59.561335 23102 ts_tablet_manager.cc:1918] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:59.561388 23102 log.cc:1196] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/9a6f50c9cf8943ccb6036592df5f7a0b
I20260926 22:44:59.561658 23102 ts_tablet_manager.cc:1939] T 9a6f50c9cf8943ccb6036592df5f7a0b P 9c80b98fd6614d978b25167f9104f3ef: Deleting consensus metadata
I20260926 22:44:59.561718 23083 tablet_bootstrap.cc:654] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.561798 23079 tablet_bootstrap.cc:654] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.562057 22556 catalog_manager.cc:4841] TS 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031): tablet 9a6f50c9cf8943ccb6036592df5f7a0b (table test-table [id=d8215a41faf74a4bb7e1ed845ffd232a]) successfully deleted
I20260926 22:44:59.562363 22848 tablet_service.cc:1467] Processing CreateTablet for tablet 74494f0897194eb5b7d4ba7ebd3c215b (DEFAULT_TABLE table=test-table [id=15d139da6ee948e89e23ae54cbbcf041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.562493 22848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74494f0897194eb5b7d4ba7ebd3c215b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.563066 23079 tablet_bootstrap.cc:492] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: No bootstrap required, opened a new log
I20260926 22:44:59.563117 23079 ts_tablet_manager.cc:1397] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:44:59.563271 23104 ts_tablet_manager.cc:1918] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:59.563287 23079 raft_consensus.cc:348] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.563346 23104 log.cc:1196] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/9a6f50c9cf8943ccb6036592df5f7a0b
I20260926 22:44:59.563383 23079 raft_consensus.cc:374] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.563412 23079 raft_consensus.cc:729] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bb79a203224c0a9eddcc5c80d9c5b7, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.563457 23079 consensus_queue.cc:260] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.563557 23079 ts_tablet_manager.cc:1428] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.563619 23104 ts_tablet_manager.cc:1939] T 9a6f50c9cf8943ccb6036592df5f7a0b P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting consensus metadata
I20260926 22:44:59.563890 23080 tablet_bootstrap.cc:492] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Bootstrap starting.
I20260926 22:44:59.564071 23103 ts_tablet_manager.cc:1905] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:59.564162 23083 tablet_bootstrap.cc:492] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: No bootstrap required, opened a new log
I20260926 22:44:59.564198 23083 ts_tablet_manager.cc:1397] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 22:44:59.564299 23080 tablet_bootstrap.cc:654] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.564344 23083 raft_consensus.cc:348] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.564419 23083 raft_consensus.cc:374] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.564440 23083 raft_consensus.cc:729] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c80b98fd6614d978b25167f9104f3ef, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.564484 23083 consensus_queue.cc:260] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.564579 23083 ts_tablet_manager.cc:1428] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:59.564754 22555 catalog_manager.cc:4841] TS a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289): tablet 9a6f50c9cf8943ccb6036592df5f7a0b (table test-table [id=d8215a41faf74a4bb7e1ed845ffd232a]) successfully deleted
I20260926 22:44:59.565460 23080 tablet_bootstrap.cc:492] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: No bootstrap required, opened a new log
I20260926 22:44:59.565526 23080 ts_tablet_manager.cc:1397] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 22:44:59.565549 23103 ts_tablet_manager.cc:1918] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:44:59.565621 23103 log.cc:1196] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/9a6f50c9cf8943ccb6036592df5f7a0b
I20260926 22:44:59.565675 23080 raft_consensus.cc:348] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.565740 23080 raft_consensus.cc:374] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.565762 23080 raft_consensus.cc:729] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.565814 23080 consensus_queue.cc:260] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.565867 23103 ts_tablet_manager.cc:1939] T 9a6f50c9cf8943ccb6036592df5f7a0b P dee8e9d391e248a8b9df535e47c4a49f: Deleting consensus metadata
I20260926 22:44:59.565901 23080 ts_tablet_manager.cc:1428] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.566220 22557 catalog_manager.cc:4841] TS dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427): tablet 9a6f50c9cf8943ccb6036592df5f7a0b (table test-table [id=d8215a41faf74a4bb7e1ed845ffd232a]) successfully deleted
I20260926 22:44:59.616276 23086 raft_consensus.cc:482] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:59.616374 23086 raft_consensus.cc:504] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.616721 23086 leader_election.cc:288] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:44:59.620147 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.616884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:34427 (local address 127.14.184.193:41955)
0926 22:44:59.616944 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.616956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.617001 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.617221 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.617222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.617224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.617298 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.617300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.618213 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.618213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.619314 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.619316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.619864 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.619865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.619956 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.619962 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.620030 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.620075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":4}
I20260926 22:44:59.620147 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.617003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:41955 (local address 127.14.184.194:34427)
0926 22:44:59.617051 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.617053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.617063 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:44:59.617093 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.617094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.617103 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.617158 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.617392 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.618129 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.619426 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.619780 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.619868 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.619877 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.620050 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.620086 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 22:44:59.620436 22868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74494f0897194eb5b7d4ba7ebd3c215b" candidate_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dee8e9d391e248a8b9df535e47c4a49f" is_pre_election: true
I20260926 22:44:59.620550 22868 raft_consensus.cc:2454] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bb79a203224c0a9eddcc5c80d9c5b7 in term 0.
I20260926 22:44:59.620694 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.617168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:49443 (local address 127.14.184.195:41031)
0926 22:44:59.617242 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.617243 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.617249 (+ 6us) server_negotiation.cc:407] Connection header received
0926 22:44:59.617268 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.617269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.617276 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.617340 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.617518 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.617967 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.619451 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.619992 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.620067 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.620075 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.620581 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.620624 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 22:44:59.620765 22661 leader_election.cc:302] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bb79a203224c0a9eddcc5c80d9c5b7, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.620893 23086 raft_consensus.cc:2781] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:59.620915 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.616901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:41031 (local address 127.14.184.193:49443)
0926 22:44:59.617037 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.617048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.617101 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.617346 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.617347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.617349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.617439 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.617440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.618049 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.618050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.619314 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.619316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.620125 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.620126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.620486 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.620491 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.620793 (+ 302us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.620836 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260926 22:44:59.620929 23086 raft_consensus.cc:482] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:59.620985 23086 raft_consensus.cc:3037] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.621263 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74494f0897194eb5b7d4ba7ebd3c215b" candidate_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef" is_pre_election: true
I20260926 22:44:59.621398 23008 raft_consensus.cc:2454] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bb79a203224c0a9eddcc5c80d9c5b7 in term 0.
I20260926 22:44:59.621888 23086 raft_consensus.cc:504] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.622030 23086 leader_election.cc:288] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [CANDIDATE]: Term 1 election: Requested vote from peers dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:44:59.622164 22868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74494f0897194eb5b7d4ba7ebd3c215b" candidate_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dee8e9d391e248a8b9df535e47c4a49f"
I20260926 22:44:59.622246 22868 raft_consensus.cc:3037] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.622908 22868 raft_consensus.cc:2454] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bb79a203224c0a9eddcc5c80d9c5b7 in term 1.
I20260926 22:44:59.623026 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74494f0897194eb5b7d4ba7ebd3c215b" candidate_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef"
I20260926 22:44:59.623099 23008 raft_consensus.cc:3037] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.623090 22661 leader_election.cc:302] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bb79a203224c0a9eddcc5c80d9c5b7, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.623162 23086 raft_consensus.cc:2781] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:59.623263 23086 raft_consensus.cc:686] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 LEADER]: Becoming Leader. State: Replica: a9bb79a203224c0a9eddcc5c80d9c5b7, State: Running, Role: LEADER
I20260926 22:44:59.623327 23086 consensus_queue.cc:237] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.623930 23008 raft_consensus.cc:2454] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bb79a203224c0a9eddcc5c80d9c5b7 in term 1.
I20260926 22:44:59.624037 22571 catalog_manager.cc:5495] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } health_report { overall_health: UNKNOWN } } }
W20260926 22:44:59.624379 22775 tablet.cc:2369] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462698, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.254@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:59.659235 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.651482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45623 (local address 127.0.0.1:40724)
0926 22:44:59.651671 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.651683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.651750 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.651896 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.651896 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.652276 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:44:59.652322 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.652324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.653340 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.653341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.653915 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.653917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.654380 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.654380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.654454 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.655059 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:44:59.655063 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:44:59.656558 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:44:59.658197 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.658204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.658205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.658405 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.658614 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:44:59.658615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:44:59.658615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:44:59.658667 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:44:59.658972 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:44:59.658978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:44:59.659019 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.659071 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.659162 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260926 22:44:59.659337 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.651573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.14.184.254:45623)
0926 22:44:59.651641 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.651643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.651755 (+ 112us) server_negotiation.cc:407] Connection header received
0926 22:44:59.651792 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.651793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.651805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.651881 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.652417 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.653234 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.654028 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.654297 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.654373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.656643 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.656644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.656644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.656652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.658120 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.658480 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.658481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.658481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.658544 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.658768 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.658768 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.658769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.658901 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.658977 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.659110 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.659224 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":36}
I20260926 22:44:59.661317 22571 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:40724:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 22:44:59.661553 22571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:59.662958 22708 tablet_service.cc:1467] Processing CreateTablet for tablet a25f054a9abb48e4a31c26064dab1094 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f54fe1908c54da592244506d21b004a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:59.663044 22848 tablet_service.cc:1467] Processing CreateTablet for tablet a25f054a9abb48e4a31c26064dab1094 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f54fe1908c54da592244506d21b004a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:59.663110 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25f054a9abb48e4a31c26064dab1094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.663193 22848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25f054a9abb48e4a31c26064dab1094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.664326 23080 tablet_bootstrap.cc:492] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: Bootstrap starting.
I20260926 22:44:59.664551 22981 tablet_service.cc:1467] Processing CreateTablet for tablet a25f054a9abb48e4a31c26064dab1094 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f54fe1908c54da592244506d21b004a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 22:44:59.664669 22981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25f054a9abb48e4a31c26064dab1094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.664708 23080 tablet_bootstrap.cc:654] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.665308 23080 tablet_bootstrap.cc:492] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: No bootstrap required, opened a new log
I20260926 22:44:59.665382 23080 ts_tablet_manager.cc:1397] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:59.665555 23080 raft_consensus.cc:348] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.665630 23080 raft_consensus.cc:374] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.665658 23080 raft_consensus.cc:729] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.665714 23080 consensus_queue.cc:260] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.665817 23080 ts_tablet_manager.cc:1428] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.665906 23084 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.666025 23084 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.666031 23079 tablet_bootstrap.cc:492] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: Bootstrap starting.
I20260926 22:44:59.666416 23079 tablet_bootstrap.cc:654] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.666741 23083 tablet_bootstrap.cc:492] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: Bootstrap starting.
I20260926 22:44:59.667119 23083 tablet_bootstrap.cc:654] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.667773 23083 tablet_bootstrap.cc:492] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: No bootstrap required, opened a new log
I20260926 22:44:59.667822 23083 ts_tablet_manager.cc:1397] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:59.667973 23083 raft_consensus.cc:348] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.668046 23083 raft_consensus.cc:374] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.668064 23083 raft_consensus.cc:729] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c80b98fd6614d978b25167f9104f3ef, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.668103 23083 consensus_queue.cc:260] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.668185 23083 ts_tablet_manager.cc:1428] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 22:44:59.668272 23097 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.668329 23097 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.668457 23079 tablet_bootstrap.cc:492] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: No bootstrap required, opened a new log
I20260926 22:44:59.668514 23079 ts_tablet_manager.cc:1397] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 22:44:59.668641 23079 raft_consensus.cc:348] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.668715 23079 raft_consensus.cc:374] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.668740 23079 raft_consensus.cc:729] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bb79a203224c0a9eddcc5c80d9c5b7, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.668778 23079 consensus_queue.cc:260] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.668864 23079 ts_tablet_manager.cc:1428] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.668931 23109 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.669078 23109 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.681988 23084 raft_consensus.cc:482] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:59.682066 23084 raft_consensus.cc:504] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.682379 23084 leader_election.cc:288] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
W20260926 22:44:59.684065 23055 tablet.cc:2369] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:44:59.686084 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.682587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:53149 (local address 127.14.184.195:41031)
0926 22:44:59.682717 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.682719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.682726 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:44:59.682750 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.682751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.682759 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.682818 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.682977 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.683727 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.685181 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.685725 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.685805 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.685814 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.685970 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.686012 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260926 22:44:59.686087 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.682532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:41031 (local address 127.14.184.194:53149)
0926 22:44:59.682585 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.682597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.682637 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.682831 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.682832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.682834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.682902 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.682904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.683812 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.683813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.685085 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.685088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.685807 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.685808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.685869 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.685872 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.685950 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.685993 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:44:59.686195 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.682545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:46289 (local address 127.14.184.194:41053)
0926 22:44:59.682722 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.682731 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.682786 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.682926 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.682926 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.682928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:44:59.682994 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.682995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.683812 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.683813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.685219 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.685221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.685944 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.685945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.686017 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.686021 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.686072 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.686106 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260926 22:44:59.686221 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.682599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:41053 (local address 127.14.184.193:46289)
0926 22:44:59.682659 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.682661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.682773 (+ 112us) server_negotiation.cc:407] Connection header received
0926 22:44:59.682853 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.682854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.682862 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.682915 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:44:59.683051 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.683729 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.685310 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.685848 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.685913 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.685919 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.686092 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.686138 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:44:59.686316 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25f054a9abb48e4a31c26064dab1094" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef" is_pre_election: true
I20260926 22:44:59.686388 23008 raft_consensus.cc:2454] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 0.
I20260926 22:44:59.686502 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25f054a9abb48e4a31c26064dab1094" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" is_pre_election: true
I20260926 22:44:59.686585 22728 raft_consensus.cc:2454] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 0.
I20260926 22:44:59.686573 22800 leader_election.cc:302] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c80b98fd6614d978b25167f9104f3ef, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.686738 23084 raft_consensus.cc:2781] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:59.686792 23084 raft_consensus.cc:482] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:59.686812 23084 raft_consensus.cc:3037] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.687526 23084 raft_consensus.cc:504] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.687660 23084 leader_election.cc:288] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 election: Requested vote from peers a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:44:59.687814 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25f054a9abb48e4a31c26064dab1094" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef"
I20260926 22:44:59.687806 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25f054a9abb48e4a31c26064dab1094" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
I20260926 22:44:59.687878 22728 raft_consensus.cc:3037] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.687878 23008 raft_consensus.cc:3037] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.688580 23008 raft_consensus.cc:2454] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 1.
I20260926 22:44:59.688581 22728 raft_consensus.cc:2454] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 1.
I20260926 22:44:59.688738 22800 leader_election.cc:302] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c80b98fd6614d978b25167f9104f3ef, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.688844 23084 raft_consensus.cc:2781] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:59.689006 23084 raft_consensus.cc:686] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [term 1 LEADER]: Becoming Leader. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Running, Role: LEADER
I20260926 22:44:59.689091 23084 consensus_queue.cc:237] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.689610 23121 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: TxnStatusTablet state changed. Reason: New leader dee8e9d391e248a8b9df535e47c4a49f. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.689689 23121 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:59.689786 22571 catalog_manager.cc:5495] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f reported cstate change: term changed from 0 to 1, leader changed from <none> to dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194). New cstate: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } health_report { overall_health: UNKNOWN } } }
I20260926 22:44:59.689900 23123 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 22:44:59.689998 23123 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 22:44:59.697845 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.695467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.14.184.194:34427)
0926 22:44:59.695510 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.695512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.695521 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:59.695612 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.695613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.695631 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.695693 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:59.695818 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.696272 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.697030 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.697233 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.697315 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.697513 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.697610 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.697685 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.697741 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.697759 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:59.697856 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.695417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:34427 (local address 127.0.0.1:38308)
0926 22:44:59.695473 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.695485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.695535 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.695697 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.695698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.695699 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:59.695754 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.695756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.696357 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.696359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.696931 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.696932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.697318 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.697319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.697394 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.697408 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.697682 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.697692 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.697738 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.697787 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:44:59.698927 22728 raft_consensus.cc:1261] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Refusing update from remote peer dee8e9d391e248a8b9df535e47c4a49f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:59.698968 23008 raft_consensus.cc:1261] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Refusing update from remote peer dee8e9d391e248a8b9df535e47c4a49f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:59.699184 23084 consensus_queue.cc:1035] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:59.699309 23121 consensus_queue.cc:1035] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:59.699743 23109 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: TxnStatusTablet state changed. Reason: New leader dee8e9d391e248a8b9df535e47c4a49f. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.699838 23109 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.699855 23097 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: TxnStatusTablet state changed. Reason: New leader dee8e9d391e248a8b9df535e47c4a49f. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.699932 23097 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.700824 23084 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.700924 23084 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:59.701246 23097 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.701336 23097 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P 9c80b98fd6614d978b25167f9104f3ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.701877 23084 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.701967 23084 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P dee8e9d391e248a8b9df535e47c4a49f: This TxnStatusTablet replica's current role is: LEADER
I20260926 22:44:59.702133 23094 mvcc.cc:205] Tried to move back new op lower bound from 7333735217964802048 to 7333735217927057408. Current Snapshot: MvccSnapshot[applied={T|T < 7333735217964802048 or (T in {7333735217964802048})}]
I20260926 22:44:59.702203 23109 tablet_replica.cc:439] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } } }
I20260926 22:44:59.702283 23109 tablet_replica.cc:442] T a25f054a9abb48e4a31c26064dab1094 P a9bb79a203224c0a9eddcc5c80d9c5b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 22:44:59.704939 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.702343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.14.184.193:46289)
0926 22:44:59.702379 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.702381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.702390 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:44:59.702442 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.702443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.702465 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.702529 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:59.702656 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.703418 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.704162 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.704463 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.704530 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.704623 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.704735 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.704805 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.704827 (+ 22us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.704846 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:59.710013 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.707408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:47039 (local address 127.14.184.194:34427)
0926 22:44:59.707444 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.707446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.707609 (+ 163us) server_negotiation.cc:407] Connection header received
0926 22:44:59.707632 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.707633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.707642 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.707692 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:59.707791 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.708499 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.709287 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.709592 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.709662 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.709798 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.709830 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.709882 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.709929 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.709948 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 22:44:59.710202 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.707344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:34427 (local address 127.14.184.193:47039)
0926 22:44:59.707505 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.707515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.707572 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.707702 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.707703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.707704 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:59.707747 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.707748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.708576 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.708576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.709205 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.709207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.709666 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.709667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.709712 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.709724 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.709889 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.709890 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.709922 (+ 32us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.709954 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260926 22:44:59.712404 23008 raft_consensus.cc:1261] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Refusing update from remote peer a9bb79a203224c0a9eddcc5c80d9c5b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:59.712430 22868 raft_consensus.cc:1261] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Refusing update from remote peer a9bb79a203224c0a9eddcc5c80d9c5b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:44:59.712634 23086 consensus_queue.cc:1035] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:59.712766 23109 consensus_queue.cc:1035] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:44:59.720312 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.717861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:54189 (local address 127.14.184.193:46289)
0926 22:44:59.717904 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.717906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.717914 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:44:59.717976 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.717976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.717984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.718019 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:44:59.718128 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.718546 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.719675 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.719866 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.719923 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.720132 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.720156 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.720193 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.720253 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.720267 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:44:59.720360 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.717804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:46289 (local address 127.14.184.194:54189)
0926 22:44:59.717855 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:44:59.717866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:44:59.717909 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:44:59.718022 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:44:59.718023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:44:59.718024 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:44:59.718075 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.718076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.718614 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.718615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.719588 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:44:59.719591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.719949 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.719950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:44:59.720024 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.720042 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:44:59.720198 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:44:59.720199 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 22:44:59.720253 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 22:44:59.720292 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 22:44:59.729835 22571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:59.729947 22571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:44:59.730535 22571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Sending DeleteTablet for 3 replicas of tablet 74494f0897194eb5b7d4ba7ebd3c215b
I20260926 22:44:59.730800 22981 tablet_service.cc:1514] Processing DeleteTablet for tablet 74494f0897194eb5b7d4ba7ebd3c215b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:59 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39548
I20260926 22:44:59.730830 22708 tablet_service.cc:1514] Processing DeleteTablet for tablet 74494f0897194eb5b7d4ba7ebd3c215b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:59 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:59802
I20260926 22:44:59.730898 23102 tablet_replica.cc:330] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: stopping tablet replica
I20260926 22:44:59.730952 23102 raft_consensus.cc:2229] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:59.730988 23102 raft_consensus.cc:2258] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:59.731029 22848 tablet_service.cc:1514] Processing DeleteTablet for tablet 74494f0897194eb5b7d4ba7ebd3c215b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:44:59 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38300
I20260926 22:44:59.731052 23104 tablet_replica.cc:330] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: stopping tablet replica
I20260926 22:44:59.731105 23104 raft_consensus.cc:2229] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:44:59.731176 23104 raft_consensus.cc:2258] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:59.731307 23103 tablet_replica.cc:330] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: stopping tablet replica
I20260926 22:44:59.731364 23103 raft_consensus.cc:2229] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:44:59.731431 23103 raft_consensus.cc:2258] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:44:59.731724 23102 ts_tablet_manager.cc:1905] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:59.733557 23102 ts_tablet_manager.cc:1918] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:59.733633 23102 log.cc:1196] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/74494f0897194eb5b7d4ba7ebd3c215b
I20260926 22:44:59.733860 23102 ts_tablet_manager.cc:1939] T 74494f0897194eb5b7d4ba7ebd3c215b P 9c80b98fd6614d978b25167f9104f3ef: Deleting consensus metadata
I20260926 22:44:59.734294 23104 ts_tablet_manager.cc:1905] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:59.734915 22556 catalog_manager.cc:4841] TS 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031): tablet 74494f0897194eb5b7d4ba7ebd3c215b (table test-table [id=15d139da6ee948e89e23ae54cbbcf041]) successfully deleted
I20260926 22:44:59.736161 23104 ts_tablet_manager.cc:1918] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:59.736222 23104 log.cc:1196] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/74494f0897194eb5b7d4ba7ebd3c215b
I20260926 22:44:59.736444 23104 ts_tablet_manager.cc:1939] T 74494f0897194eb5b7d4ba7ebd3c215b P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting consensus metadata
I20260926 22:44:59.736693 23103 ts_tablet_manager.cc:1905] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:44:59.738334 23103 ts_tablet_manager.cc:1918] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:44:59.738348 22555 catalog_manager.cc:4841] TS a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289): tablet 74494f0897194eb5b7d4ba7ebd3c215b (table test-table [id=15d139da6ee948e89e23ae54cbbcf041]) successfully deleted
I20260926 22:44:59.738396 23103 log.cc:1196] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/74494f0897194eb5b7d4ba7ebd3c215b
I20260926 22:44:59.738597 23103 ts_tablet_manager.cc:1939] T 74494f0897194eb5b7d4ba7ebd3c215b P dee8e9d391e248a8b9df535e47c4a49f: Deleting consensus metadata
I20260926 22:44:59.738974 22557 catalog_manager.cc:4841] TS dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427): tablet 74494f0897194eb5b7d4ba7ebd3c215b (table test-table [id=15d139da6ee948e89e23ae54cbbcf041]) successfully deleted
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 22:44:59.764940 22640 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:44:59.759743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.14.184.254:45623)
0926 22:44:59.759801 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.759804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.759932 (+ 128us) server_negotiation.cc:407] Connection header received
0926 22:44:59.760000 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.760001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.760012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.760067 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.760616 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.761115 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.761857 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.762054 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.762130 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.764891 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40726: BlockingRecv error: recv got EOF from 127.0.0.1:40726 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 26 22:44:59 dist-test-slave-3986 krb5kdc[22526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM
I20260926 22:44:59.772924 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:44:59.765913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.14.184.254:45623)
0926 22:44:59.765951 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 22:44:59.765953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:44:59.766158 (+ 205us) server_negotiation.cc:407] Connection header received
0926 22:44:59.766256 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:44:59.766257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:44:59.766268 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:44:59.766324 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:44:59.766811 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.767277 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.768028 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:44:59.768260 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:44:59.768343 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:44:59.770394 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:44:59.770395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:44:59.770396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:44:59.770403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:44:59.771835 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.772176 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.772178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.772178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.772236 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:44:59.772438 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:44:59.772439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:44:59.772439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:44:59.772540 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:44:59.772642 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 22:44:59.772736 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 22:44:59.772809 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:44:59.774401 22571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:44:59.774570 22571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:44:59.775898 22708 tablet_service.cc:1467] Processing CreateTablet for tablet 689202b97ce34fef882b89c68063a135 (DEFAULT_TABLE table=test-table [id=f456a3d19f684c32b85cbc7d04fd5d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.775951 22848 tablet_service.cc:1467] Processing CreateTablet for tablet 689202b97ce34fef882b89c68063a135 (DEFAULT_TABLE table=test-table [id=f456a3d19f684c32b85cbc7d04fd5d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.776033 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689202b97ce34fef882b89c68063a135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.776047 22848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689202b97ce34fef882b89c68063a135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.777318 23079 tablet_bootstrap.cc:492] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Bootstrap starting.
I20260926 22:44:59.777318 23080 tablet_bootstrap.cc:492] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Bootstrap starting.
I20260926 22:44:59.777642 23080 tablet_bootstrap.cc:654] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.777725 23079 tablet_bootstrap.cc:654] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.778337 23079 tablet_bootstrap.cc:492] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: No bootstrap required, opened a new log
I20260926 22:44:59.778388 23079 ts_tablet_manager.cc:1397] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:59.778434 22981 tablet_service.cc:1467] Processing CreateTablet for tablet 689202b97ce34fef882b89c68063a135 (DEFAULT_TABLE table=test-table [id=f456a3d19f684c32b85cbc7d04fd5d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:44:59.778545 22981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689202b97ce34fef882b89c68063a135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:44:59.778537 23079 raft_consensus.cc:348] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.778604 23079 raft_consensus.cc:374] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.778628 23079 raft_consensus.cc:729] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bb79a203224c0a9eddcc5c80d9c5b7, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.778697 23079 consensus_queue.cc:260] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.778803 23079 ts_tablet_manager.cc:1428] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.779863 23083 tablet_bootstrap.cc:492] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Bootstrap starting.
I20260926 22:44:59.780182 23083 tablet_bootstrap.cc:654] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Neither blocks nor log segments found. Creating new log.
I20260926 22:44:59.780454 23080 tablet_bootstrap.cc:492] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: No bootstrap required, opened a new log
I20260926 22:44:59.780508 23080 ts_tablet_manager.cc:1397] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 22:44:59.780602 23080 raft_consensus.cc:348] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.780659 23080 raft_consensus.cc:374] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.780673 23080 raft_consensus.cc:729] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.780685 23083 tablet_bootstrap.cc:492] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: No bootstrap required, opened a new log
I20260926 22:44:59.780728 23083 ts_tablet_manager.cc:1397] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:44:59.780735 23080 consensus_queue.cc:260] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.780822 23080 ts_tablet_manager.cc:1428] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.780831 23083 raft_consensus.cc:348] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.780892 23083 raft_consensus.cc:374] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:44:59.780920 23083 raft_consensus.cc:729] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c80b98fd6614d978b25167f9104f3ef, State: Initialized, Role: FOLLOWER
I20260926 22:44:59.780959 23083 consensus_queue.cc:260] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.781039 23083 ts_tablet_manager.cc:1428] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:44:59.973276 23131 raft_consensus.cc:482] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:44:59.973400 23131 raft_consensus.cc:504] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.973616 23131 leader_election.cc:288] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:44:59.973861 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689202b97ce34fef882b89c68063a135" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef" is_pre_election: true
I20260926 22:44:59.973863 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689202b97ce34fef882b89c68063a135" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" is_pre_election: true
I20260926 22:44:59.973970 23008 raft_consensus.cc:2454] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 0.
I20260926 22:44:59.973968 22728 raft_consensus.cc:2454] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 0.
I20260926 22:44:59.974139 22799 leader_election.cc:302] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bb79a203224c0a9eddcc5c80d9c5b7, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.974237 23131 raft_consensus.cc:2781] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:44:59.974277 23131 raft_consensus.cc:482] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:44:59.974294 23131 raft_consensus.cc:3037] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.975025 23131 raft_consensus.cc:504] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.975168 23131 leader_election.cc:288] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 election: Requested vote from peers a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:44:59.975319 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689202b97ce34fef882b89c68063a135" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
I20260926 22:44:59.975324 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689202b97ce34fef882b89c68063a135" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef"
I20260926 22:44:59.975399 22728 raft_consensus.cc:3037] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.975423 23008 raft_consensus.cc:3037] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:44:59.975994 22728 raft_consensus.cc:2454] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 1.
I20260926 22:44:59.976084 23008 raft_consensus.cc:2454] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 1.
I20260926 22:44:59.976158 22799 leader_election.cc:302] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bb79a203224c0a9eddcc5c80d9c5b7, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:44:59.976282 23131 raft_consensus.cc:2781] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:44:59.976331 23131 raft_consensus.cc:686] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 1 LEADER]: Becoming Leader. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Running, Role: LEADER
I20260926 22:44:59.976363 23131 consensus_queue.cc:237] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:44:59.976981 22570 catalog_manager.cc:5495] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f reported cstate change: term changed from 0 to 1, leader changed from <none> to dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194). New cstate: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } health_report { overall_health: UNKNOWN } } }
I20260926 22:45:00.009706 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:00.007499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.14.184.194:34427)
0926 22:45:00.007562 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:00.007564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:00.007572 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:00.007630 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:00.007631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:00.007641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:00.007692 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:45:00.007847 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:00.008302 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:00.009015 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:00.009197 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:00.009257 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:00.009449 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:45:00.009475 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:45:00.009513 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:00.009571 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 22:45:00.009588 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 22:45:00.010720 22728 raft_consensus.cc:1261] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Refusing update from remote peer dee8e9d391e248a8b9df535e47c4a49f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:45:00.010736 23008 raft_consensus.cc:1261] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Refusing update from remote peer dee8e9d391e248a8b9df535e47c4a49f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:45:00.011010 23121 consensus_queue.cc:1035] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:45:00.011112 23131 consensus_queue.cc:1035] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:45:00.014309 22570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:45:00.014401 22570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:45:00.015015 22570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Sending DeleteTablet for 3 replicas of tablet 689202b97ce34fef882b89c68063a135
I20260926 22:45:00.015275 22708 tablet_service.cc:1514] Processing DeleteTablet for tablet 689202b97ce34fef882b89c68063a135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:00 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:59802
I20260926 22:45:00.015275 22848 tablet_service.cc:1514] Processing DeleteTablet for tablet 689202b97ce34fef882b89c68063a135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:00 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38300
I20260926 22:45:00.015395 23103 tablet_replica.cc:330] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: stopping tablet replica
I20260926 22:45:00.015460 23103 raft_consensus.cc:2229] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:45:00.015551 23103 raft_consensus.cc:2258] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:45:00.015636 23104 tablet_replica.cc:330] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: stopping tablet replica
I20260926 22:45:00.015686 23104 raft_consensus.cc:2229] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:45:00.015615 22570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 22:45:00.015726 23104 raft_consensus.cc:2258] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 22:45:00.015763 22570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:45:00.015748 22981 tablet_service.cc:1514] Processing DeleteTablet for tablet 689202b97ce34fef882b89c68063a135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:00 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39548
I20260926 22:45:00.015836 23102 tablet_replica.cc:330] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: stopping tablet replica
I20260926 22:45:00.015879 23102 raft_consensus.cc:2229] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:45:00.015919 23102 raft_consensus.cc:2258] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:45:00.016232 23104 ts_tablet_manager.cc:1905] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:00.016482 23103 ts_tablet_manager.cc:1905] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:00.017958 23103 ts_tablet_manager.cc:1918] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:45:00.017957 23104 ts_tablet_manager.cc:1918] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:45:00.017998 23103 log.cc:1196] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/689202b97ce34fef882b89c68063a135
I20260926 22:45:00.018013 23104 log.cc:1196] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/689202b97ce34fef882b89c68063a135
I20260926 22:45:00.018152 23103 ts_tablet_manager.cc:1939] T 689202b97ce34fef882b89c68063a135 P dee8e9d391e248a8b9df535e47c4a49f: Deleting consensus metadata
I20260926 22:45:00.018239 23104 ts_tablet_manager.cc:1939] T 689202b97ce34fef882b89c68063a135 P a9bb79a203224c0a9eddcc5c80d9c5b7: Deleting consensus metadata
I20260926 22:45:00.018541 22555 catalog_manager.cc:4841] TS a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289): tablet 689202b97ce34fef882b89c68063a135 (table test-table [id=f456a3d19f684c32b85cbc7d04fd5d48]) successfully deleted
I20260926 22:45:00.018852 22557 catalog_manager.cc:4841] TS dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427): tablet 689202b97ce34fef882b89c68063a135 (table test-table [id=f456a3d19f684c32b85cbc7d04fd5d48]) successfully deleted
I20260926 22:45:00.019178 22848 tablet_service.cc:1467] Processing CreateTablet for tablet f7179c538a844780b99cc8628086650f (DEFAULT_TABLE table=test-table [id=9abaf9a51c494fb7b97c798e0de1138b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:45:00.019256 23102 ts_tablet_manager.cc:1905] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:00.019290 22848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7179c538a844780b99cc8628086650f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:00.020442 23080 tablet_bootstrap.cc:492] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Bootstrap starting.
I20260926 22:45:00.020470 23102 ts_tablet_manager.cc:1918] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:45:00.020514 23102 log.cc:1196] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/689202b97ce34fef882b89c68063a135
I20260926 22:45:00.020614 22708 tablet_service.cc:1467] Processing CreateTablet for tablet f7179c538a844780b99cc8628086650f (DEFAULT_TABLE table=test-table [id=9abaf9a51c494fb7b97c798e0de1138b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:45:00.020710 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7179c538a844780b99cc8628086650f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:00.020725 23102 ts_tablet_manager.cc:1939] T 689202b97ce34fef882b89c68063a135 P 9c80b98fd6614d978b25167f9104f3ef: Deleting consensus metadata
I20260926 22:45:00.020756 23080 tablet_bootstrap.cc:654] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:00.021368 23080 tablet_bootstrap.cc:492] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: No bootstrap required, opened a new log
I20260926 22:45:00.021435 23080 ts_tablet_manager.cc:1397] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:45:00.021550 23080 raft_consensus.cc:348] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.021638 23080 raft_consensus.cc:374] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:00.021665 23080 raft_consensus.cc:729] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Initialized, Role: FOLLOWER
I20260926 22:45:00.021705 23080 consensus_queue.cc:260] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.021791 23079 tablet_bootstrap.cc:492] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7: Bootstrap starting.
I20260926 22:45:00.021800 23080 ts_tablet_manager.cc:1428] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:00.022014 22556 catalog_manager.cc:4841] TS 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031): tablet 689202b97ce34fef882b89c68063a135 (table test-table [id=f456a3d19f684c32b85cbc7d04fd5d48]) successfully deleted
I20260926 22:45:00.022125 23079 tablet_bootstrap.cc:654] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:00.022226 22981 tablet_service.cc:1467] Processing CreateTablet for tablet f7179c538a844780b99cc8628086650f (DEFAULT_TABLE table=test-table [id=9abaf9a51c494fb7b97c798e0de1138b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:45:00.022315 22981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7179c538a844780b99cc8628086650f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:00.022847 23079 tablet_bootstrap.cc:492] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7: No bootstrap required, opened a new log
I20260926 22:45:00.022919 23079 ts_tablet_manager.cc:1397] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:45:00.023032 23079 raft_consensus.cc:348] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.023115 23079 raft_consensus.cc:374] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:00.023142 23079 raft_consensus.cc:729] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bb79a203224c0a9eddcc5c80d9c5b7, State: Initialized, Role: FOLLOWER
I20260926 22:45:00.023185 23079 consensus_queue.cc:260] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.023283 23079 ts_tablet_manager.cc:1428] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:00.023484 23083 tablet_bootstrap.cc:492] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Bootstrap starting.
I20260926 22:45:00.023804 23083 tablet_bootstrap.cc:654] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:00.024408 23083 tablet_bootstrap.cc:492] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: No bootstrap required, opened a new log
I20260926 22:45:00.024468 23083 ts_tablet_manager.cc:1397] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:45:00.024569 23083 raft_consensus.cc:348] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.024647 23083 raft_consensus.cc:374] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:00.024680 23083 raft_consensus.cc:729] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c80b98fd6614d978b25167f9104f3ef, State: Initialized, Role: FOLLOWER
I20260926 22:45:00.024735 23083 consensus_queue.cc:260] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.024824 23083 ts_tablet_manager.cc:1428] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:00.032203 23121 raft_consensus.cc:482] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:45:00.032285 23121 raft_consensus.cc:504] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.032418 23121 leader_election.cc:288] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:45:00.032552 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7179c538a844780b99cc8628086650f" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" is_pre_election: true
I20260926 22:45:00.032665 22728 raft_consensus.cc:2454] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 0.
I20260926 22:45:00.032765 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7179c538a844780b99cc8628086650f" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef" is_pre_election: true
I20260926 22:45:00.032799 22799 leader_election.cc:302] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bb79a203224c0a9eddcc5c80d9c5b7, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:45:00.032845 23008 raft_consensus.cc:2454] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 0.
I20260926 22:45:00.032871 23121 raft_consensus.cc:2781] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:45:00.032902 23121 raft_consensus.cc:482] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:45:00.032932 23121 raft_consensus.cc:3037] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:00.033643 23121 raft_consensus.cc:504] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.033766 23121 leader_election.cc:288] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 election: Requested vote from peers a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289), 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031)
I20260926 22:45:00.034001 23008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7179c538a844780b99cc8628086650f" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c80b98fd6614d978b25167f9104f3ef"
I20260926 22:45:00.034071 23008 raft_consensus.cc:3037] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:00.034089 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7179c538a844780b99cc8628086650f" candidate_uuid: "dee8e9d391e248a8b9df535e47c4a49f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7"
I20260926 22:45:00.034163 22728 raft_consensus.cc:3037] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:00.034761 23008 raft_consensus.cc:2454] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 1.
I20260926 22:45:00.034888 22728 raft_consensus.cc:2454] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dee8e9d391e248a8b9df535e47c4a49f in term 1.
I20260926 22:45:00.035053 22800 leader_election.cc:302] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c80b98fd6614d978b25167f9104f3ef, dee8e9d391e248a8b9df535e47c4a49f; no voters:
I20260926 22:45:00.035151 23121 raft_consensus.cc:2781] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:45:00.035212 23121 raft_consensus.cc:686] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 1 LEADER]: Becoming Leader. State: Replica: dee8e9d391e248a8b9df535e47c4a49f, State: Running, Role: LEADER
I20260926 22:45:00.035259 23121 consensus_queue.cc:237] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } }
I20260926 22:45:00.035713 22571 catalog_manager.cc:5495] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f reported cstate change: term changed from 0 to 1, leader changed from <none> to dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194). New cstate: current_term: 1 leader_uuid: "dee8e9d391e248a8b9df535e47c4a49f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dee8e9d391e248a8b9df535e47c4a49f" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 34427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 } health_report { overall_health: UNKNOWN } } }
I20260926 22:45:00.047722 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:00.045397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:47381 (local address 127.14.184.194:34427)
0926 22:45:00.045431 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:00.045433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:00.045442 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:45:00.045495 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:00.045496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:00.045504 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:00.045544 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:45:00.045652 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:00.046093 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:00.047117 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:00.047288 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:00.047340 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:00.047484 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:45:00.047515 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:45:00.047565 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:00.047637 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 22:45:00.047659 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":15}
I20260926 22:45:00.047796 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:00.045339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:34427 (local address 127.14.184.194:47381)
0926 22:45:00.045388 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:00.045400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:00.045435 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:00.045556 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:00.045557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:00.045558 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 22:45:00.045600 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:00.045602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:00.046155 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:00.046156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:00.047045 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:00.047047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:00.047356 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:00.047357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:00.047412 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:00.047416 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 22:45:00.047582 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 22:45:00.047584 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:45:00.047636 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:45:00.047686 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":3}
I20260926 22:45:00.049566 23008 raft_consensus.cc:1261] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Refusing update from remote peer dee8e9d391e248a8b9df535e47c4a49f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:45:00.049685 22728 raft_consensus.cc:1261] T f7179c538a844780b99cc8628086650f P a9bb79a203224c0a9eddcc5c80d9c5b7 [term 1 FOLLOWER]: Refusing update from remote peer dee8e9d391e248a8b9df535e47c4a49f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:45:00.049762 23131 consensus_queue.cc:1035] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c80b98fd6614d978b25167f9104f3ef" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:45:00.049865 23121 consensus_queue.cc:1035] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bb79a203224c0a9eddcc5c80d9c5b7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:45:00.060209 22570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:45:00.060319 22570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:45:00.060897 22570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 775214a523514a0289b14f62e255e87b: Sending DeleteTablet for 3 replicas of tablet f7179c538a844780b99cc8628086650f
I20260926 22:45:00.061203 22708 tablet_service.cc:1514] Processing DeleteTablet for tablet f7179c538a844780b99cc8628086650f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:00 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:59802
I20260926 22:45:00.061301 22981 tablet_service.cc:1514] Processing DeleteTablet for tablet f7179c538a844780b99cc8628086650f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:00 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:39548
I20260926 22:45:00.061379 23102 tablet_replica.cc:330] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: stopping tablet replica
I20260926 22:45:00.061421 23102 raft_consensus.cc:2229] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:45:00.061460 23102 raft_consensus.cc:2258] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:45:00.061594 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 22643
I20260926 22:45:00.062454 22848 tablet_service.cc:1514] Processing DeleteTablet for tablet f7179c538a844780b99cc8628086650f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:00 UTC) from {username='oryx', principal='oryx/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38300
I20260926 22:45:00.062529 23103 tablet_replica.cc:330] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: stopping tablet replica
I20260926 22:45:00.062567 23103 raft_consensus.cc:2229] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:45:00.062614 23103 raft_consensus.cc:2258] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:45:00.062956 23103 ts_tablet_manager.cc:1905] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:00.064504 23103 ts_tablet_manager.cc:1918] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:45:00.064575 23103 log.cc:1196] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/f7179c538a844780b99cc8628086650f
I20260926 22:45:00.064577 23102 ts_tablet_manager.cc:1905] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:00.064785 23103 ts_tablet_manager.cc:1939] T f7179c538a844780b99cc8628086650f P dee8e9d391e248a8b9df535e47c4a49f: Deleting consensus metadata
I20260926 22:45:00.065119 22557 catalog_manager.cc:4841] TS dee8e9d391e248a8b9df535e47c4a49f (127.14.184.194:34427): tablet f7179c538a844780b99cc8628086650f (table test-table [id=9abaf9a51c494fb7b97c798e0de1138b]) successfully deleted
I20260926 22:45:00.065935 23102 ts_tablet_manager.cc:1918] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 22:45:00.065995 23102 log.cc:1196] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/f7179c538a844780b99cc8628086650f
I20260926 22:45:00.066200 23102 ts_tablet_manager.cc:1939] T f7179c538a844780b99cc8628086650f P 9c80b98fd6614d978b25167f9104f3ef: Deleting consensus metadata
I20260926 22:45:00.066768 22556 catalog_manager.cc:4841] TS 9c80b98fd6614d978b25167f9104f3ef (127.14.184.195:41031): tablet f7179c538a844780b99cc8628086650f (table test-table [id=9abaf9a51c494fb7b97c798e0de1138b]) successfully deleted
W20260926 22:45:00.068362 22555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:46289 (error 108)
I20260926 22:45:00.068609 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 22783
I20260926 22:45:00.070051 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:00.068790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:46289 (local address 127.0.0.1:59810)
0926 22:45:00.069977 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:00.070020 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:46289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":37,"threads_started":1}
W20260926 22:45:00.070170 22555 catalog_manager.cc:4584] TS a9bb79a203224c0a9eddcc5c80d9c5b7 (127.14.184.193:46289): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7179c538a844780b99cc8628086650f: Network error: Client connection negotiation failed: client connection to 127.14.184.193:46289: connect: Connection refused (error 111)
I20260926 22:45:00.074457 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 22923
I20260926 22:45:00.079758 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 22539
2026-09-26T22:45:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2391 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:45:00 dist-test-slave-3986 krb5kdc[23170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:45:00 dist-test-slave-3986 krb5kdc[23170](info): set up 2 sockets
Sep 26 22:45:00 dist-test-slave-3986 krb5kdc[23170](info): commencing operation
krb5kdc: starting...
W20260926 22:45:01.107437 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:45:01 dist-test-slave-3986 krb5kdc[23170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:45:01.218961 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37545
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:01.298872 23183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:01.299070 23183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:01.299093 23183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:01.300266 23183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:45:01.300305 23183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:01.300320 23183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:01.300371 23183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:45:01.300388 23183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:45:01.301823 23183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:37545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:01.302093 23183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:01.302328 23183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:01.307142 23189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:01.307199 23183 server_base.cc:1034] running on GCE node
W20260926 22:45:01.307147 23191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:01.307147 23188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:01.307571 23183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:01.307857 23183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:01.309015 23183 hybrid_clock.cc:648] HybridClock initialized: now 1790462701308994 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T22:45:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1235 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:45:01 dist-test-slave-3986 krb5kdc[23196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:45:01 dist-test-slave-3986 krb5kdc[23196](info): set up 2 sockets
Sep 26 22:45:01 dist-test-slave-3986 krb5kdc[23196](info): commencing operation
krb5kdc: starting...
W20260926 22:45:02.383486 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:45:02 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:45:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:45:02.501157 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43753
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:02.580566 23209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:02.580792 23209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:02.580828 23209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:02.581975 23209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:45:02.582042 23209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:02.582067 23209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:02.582087 23209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:45:02.582118 23209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:45:02.583580 23209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:02.583845 23209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:02.584059 23209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:02.588768 23214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:02.588757 23215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:02.588824 23209 server_base.cc:1034] running on GCE node
W20260926 22:45:02.588780 23217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:02.589296 23209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:02.589565 23209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:02.590723 23209 hybrid_clock.cc:648] HybridClock initialized: now 1790462702590708 us; error 25 us; skew 500 ppm
Sep 26 22:45:02 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:02.597883 23209 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:45:02.598299 23209 webserver.cc:460] Webserver started at http://127.14.184.254:32935/ using document root <none> and password file <none>
I20260926 22:45:02.598511 23209 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:02.598572 23209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:02.598764 23209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:02.599784 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "57db547a1baa4b1daa851e19171ceb2a"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "8c70962c70058cd433160a837966c86a"
server_key_iv: "468e46cde80a2ae861f8ce9a266fdc85"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.600138 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "57db547a1baa4b1daa851e19171ceb2a"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "8c70962c70058cd433160a837966c86a"
server_key_iv: "468e46cde80a2ae861f8ce9a266fdc85"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.601421 23209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:45:02.602144 23224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:02.602334 23209 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:02.602418 23209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "57db547a1baa4b1daa851e19171ceb2a"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "8c70962c70058cd433160a837966c86a"
server_key_iv: "468e46cde80a2ae861f8ce9a266fdc85"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.602470 23209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:02.631109 23209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:02.633044 23209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:02.633149 23209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:02.636950 23209 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43753
I20260926 22:45:02.637023 23276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43753 every 8 connection(s)
I20260926 22:45:02.637359 23209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:45:02.638214 23277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:02.640487 23277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Bootstrap starting.
I20260926 22:45:02.641042 23277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:02.641330 23277 log.cc:824] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:02.641990 23277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: No bootstrap required, opened a new log
I20260926 22:45:02.643658 23277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:02.643790 23277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:02.643812 23277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57db547a1baa4b1daa851e19171ceb2a, State: Initialized, Role: FOLLOWER
I20260926 22:45:02.643977 23277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:02.644054 23277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:45:02.644098 23277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:45:02.644147 23277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:02.644650 23277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:02.644758 23277 leader_election.cc:302] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57db547a1baa4b1daa851e19171ceb2a; no voters:
I20260926 22:45:02.644934 23277 leader_election.cc:288] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:45:02.644973 23280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:45:02.645151 23280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 LEADER]: Becoming Leader. State: Replica: 57db547a1baa4b1daa851e19171ceb2a, State: Running, Role: LEADER
I20260926 22:45:02.645185 23277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:45:02.645232 23280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:02.645584 23282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57db547a1baa4b1daa851e19171ceb2a. Latest consensus state: current_term: 1 leader_uuid: "57db547a1baa4b1daa851e19171ceb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } } }
I20260926 22:45:02.645706 23281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57db547a1baa4b1daa851e19171ceb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } } }
I20260926 22:45:02.645746 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23209
I20260926 22:45:02.645731 23282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:02.645794 23281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:02.645885 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:45:02.645993 23289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:45:02.646085 15075 external_mini_cluster.cc:1428] Setting key a65abc065a2fa6fe193c20a9534ce240
I20260926 22:45:02.646716 23289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:45:02.648209 23289 catalog_manager.cc:1347] Generated new cluster ID: 6bbc5c78172543719dedb2abe4f23a82
I20260926 22:45:02.648258 23289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:45:02 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:02.656032 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:02.646614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.14.184.254:43753)
0926 22:45:02.646964 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:02.646974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:02.647011 (+ 37us) server_negotiation.cc:407] Connection header received
0926 22:45:02.647314 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:02.647322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:02.647394 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:02.647536 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:02.648240 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:02.649180 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:02.649964 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:02.650421 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:02.650547 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:02.652811 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:02.652815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:02.652815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:02.652827 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:02.654445 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:02.654911 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:02.654914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:02.654915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:02.655010 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:02.655227 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:02.655228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:02.655228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:02.655454 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:02.655627 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:02.655702 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 22:45:02.655838 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":95,"threads_started":1}
I20260926 22:45:02.682503 23289 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:45:02.683135 23289 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:45:02.691648 23289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Generated new TSK 0
I20260926 22:45:02.691834 23289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:45:02.721508 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43753
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:02.802450 23303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:02.802632 23303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:02.802654 23303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:02.803843 23303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:02.803884 23303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:02.803910 23303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:45:02.805451 23303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:02.805686 23303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:02.805910 23303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:02.811976 23311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:02.812062 23308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:02.812228 23303 server_base.cc:1034] running on GCE node
W20260926 22:45:02.812517 23309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:02.812767 23303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:02.813066 23303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:02.814215 23303 hybrid_clock.cc:648] HybridClock initialized: now 1790462702814197 us; error 38 us; skew 500 ppm
Sep 26 22:45:02 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:02.817564 23303 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:45:02.818095 23303 webserver.cc:460] Webserver started at http://127.14.184.193:44043/ using document root <none> and password file <none>
I20260926 22:45:02.818362 23303 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:02.818452 23303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:02.818598 23303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:02.820114 23303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "aa1f508174224caca4aafe53e510ef97"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "e1c8511635e45e4fb9942fc08fad290e"
server_key_iv: "d9224b695dfac54be7d867ff0c9d3638"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.820570 23303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "aa1f508174224caca4aafe53e510ef97"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "e1c8511635e45e4fb9942fc08fad290e"
server_key_iv: "d9224b695dfac54be7d867ff0c9d3638"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.822595 23303 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 22:45:02.823479 23318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:02.823736 23303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:45:02.823838 23303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "aa1f508174224caca4aafe53e510ef97"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "e1c8511635e45e4fb9942fc08fad290e"
server_key_iv: "d9224b695dfac54be7d867ff0c9d3638"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.823935 23303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:02.856395 23303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:02.858420 23303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:02.858582 23303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:02.858855 23303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:02.859184 23303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:02.859238 23303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:02.859279 23303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:02.859309 23303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:02.865064 23303 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:34527
I20260926 22:45:02.865113 23431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:34527 every 8 connection(s)
I20260926 22:45:02.865475 23303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:45:02.866158 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23303
I20260926 22:45:02.866274 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:45:02.866500 15075 external_mini_cluster.cc:1428] Setting key cbe27b3c1fce746593be05eaa5870324
Sep 26 22:45:02 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
I20260926 22:45:02.875640 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:02.866174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:48173 (local address 127.14.184.254:43753)
0926 22:45:02.866239 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:02.866241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:02.866455 (+ 214us) server_negotiation.cc:407] Connection header received
0926 22:45:02.867010 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:02.867011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:02.867023 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:02.867096 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:02.868383 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:02.869103 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:02.870374 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:02.870575 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:02.870628 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:02.872821 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:02.872823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:02.872824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:02.872831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:02.874141 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:02.874568 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:02.874569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:02.874570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:02.874628 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:02.874985 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:02.874986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:02.874987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:02.875100 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:02.875181 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:02.875459 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 22:45:02.875550 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:45:02.875792 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:02.866106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:48173)
0926 22:45:02.866283 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:02.866324 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:02.866926 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:02.867178 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:02.867183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:02.868022 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:02.868287 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:02.868293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:02.869196 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:02.869197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:02.870291 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:02.870294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:02.870660 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:02.870662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:02.870842 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:02.871341 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:02.871352 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:02.872726 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:02.874235 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:02.874242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:02.874244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:02.874454 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:02.874748 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:02.874748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:02.874749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:02.874889 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:02.875170 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:02.875177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:02.875314 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 22:45:02.875413 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 22:45:02.875567 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260926 22:45:02.876300 23432 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43753
I20260926 22:45:02.876391 23432 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:02.876585 23432 heartbeater.cc:511] Master 127.14.184.254:43753 requested a full tablet report, sending...
I20260926 22:45:02.877156 23241 ts_manager.cc:194] Registered new tserver with Master: aa1f508174224caca4aafe53e510ef97 (127.14.184.193:34527)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
I20260926 22:45:02.877831 23241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:48173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:45:02.890878 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43753
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:02.968899 23437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:02.969084 23437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:02.969105 23437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:02.970243 23437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:02.970283 23437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:02.970307 23437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:45:02.971679 23437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:02.971900 23437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:02.972117 23437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:02.976943 23443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:02.976989 23442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:02.977005 23445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:02.977164 23437 server_base.cc:1034] running on GCE node
I20260926 22:45:02.977353 23437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:02.977532 23437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:02.978698 23437 hybrid_clock.cc:648] HybridClock initialized: now 1790462702978655 us; error 28 us; skew 500 ppm
Sep 26 22:45:02 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:02.981096 23437 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:45:02.981472 23437 webserver.cc:460] Webserver started at http://127.14.184.194:41925/ using document root <none> and password file <none>
I20260926 22:45:02.981668 23437 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:02.981717 23437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:02.981853 23437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:02.982882 23437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "dd7da669591d404d8b7e3eac580c9da4"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "6badbb52c43fd0dcedb31e070dae5b39"
server_key_iv: "78b7084751d8ef1e38a7197f0808a40c"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.983232 23437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "dd7da669591d404d8b7e3eac580c9da4"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "6badbb52c43fd0dcedb31e070dae5b39"
server_key_iv: "78b7084751d8ef1e38a7197f0808a40c"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.984474 23437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:45:02.985203 23452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:02.985406 23437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:45:02.985496 23437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "dd7da669591d404d8b7e3eac580c9da4"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "6badbb52c43fd0dcedb31e070dae5b39"
server_key_iv: "78b7084751d8ef1e38a7197f0808a40c"
server_key_version: "encryptionkey@0"
I20260926 22:45:02.985567 23437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:02.997198 23437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:02.999558 23437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:02.999670 23437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:02.999840 23437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:03.000140 23437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:03.000195 23437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.000253 23437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:03.000283 23437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.006213 23437 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:35191
I20260926 22:45:03.006278 23565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:35191 every 8 connection(s)
I20260926 22:45:03.006623 23437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462702, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:03.015337 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.007391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:48293 (local address 127.14.184.254:43753)
0926 22:45:03.007450 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.007452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.007663 (+ 211us) server_negotiation.cc:407] Connection header received
0926 22:45:03.008187 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.008189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.008199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.008253 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.009137 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.009622 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.010597 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.010853 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.010906 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.012962 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:03.012964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:03.012964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:03.012970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:03.014146 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:03.014491 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:03.014491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:03.014492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:03.014539 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:03.014824 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:03.014824 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:03.014824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:03.014924 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:03.014977 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:03.015200 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 22:45:03.015261 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 22:45:03.015476 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23437
I20260926 22:45:03.015584 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.007360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:48293)
0926 22:45:03.007512 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.007547 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.008094 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.008306 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.008309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.008860 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.009049 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.009054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.009709 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.009710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.010519 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.010521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.010922 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.010923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.011021 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.011557 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.011564 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.012878 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:03.014214 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:03.014229 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:03.014231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:03.014429 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:03.014595 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:03.014595 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:03.014596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:03.014762 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:03.014981 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:03.014986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:03.015095 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 22:45:03.015175 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 22:45:03.015341 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260926 22:45:03.015599 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:45:03.015825 15075 external_mini_cluster.cc:1428] Setting key 41879178ee15faf6c799342d27847113
I20260926 22:45:03.016041 23566 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43753
I20260926 22:45:03.016155 23566 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:03.016319 23566 heartbeater.cc:511] Master 127.14.184.254:43753 requested a full tablet report, sending...
I20260926 22:45:03.016707 23241 ts_manager.cc:194] Registered new tserver with Master: dd7da669591d404d8b7e3eac580c9da4 (127.14.184.194:35191)
I20260926 22:45:03.017076 23241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:48293
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:45:03.039662 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43753
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:03.117801 23571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:03.117993 23571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:03.118017 23571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:03.119109 23571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:03.119145 23571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:03.119170 23571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:45:03.120484 23571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:03.120653 23571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:03.120826 23571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:03.125702 23577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.125698 23579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.125747 23571 server_base.cc:1034] running on GCE node
W20260926 22:45:03.125681 23576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.126094 23571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:03.126350 23571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:03.127518 23571 hybrid_clock.cc:648] HybridClock initialized: now 1790462703127514 us; error 36 us; skew 500 ppm
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462703, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:03.130116 23571 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:45:03.130539 23571 webserver.cc:460] Webserver started at http://127.14.184.195:37261/ using document root <none> and password file <none>
I20260926 22:45:03.130808 23571 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:03.130875 23571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:03.131023 23571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:03.132052 23571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "9ecd02fde6914d4dbc1e31d757b66cb5"
format_stamp: "Formatted at 2026-09-26 22:45:03 on dist-test-slave-3986"
server_key: "e5d827612c8705a1a59d270bea44326e"
server_key_iv: "8b2b533955d3ecb186d9fbe408362329"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.132416 23571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "9ecd02fde6914d4dbc1e31d757b66cb5"
format_stamp: "Formatted at 2026-09-26 22:45:03 on dist-test-slave-3986"
server_key: "e5d827612c8705a1a59d270bea44326e"
server_key_iv: "8b2b533955d3ecb186d9fbe408362329"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.133695 23571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:45:03.134456 23586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.134621 23571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:03.134747 23571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "9ecd02fde6914d4dbc1e31d757b66cb5"
format_stamp: "Formatted at 2026-09-26 22:45:03 on dist-test-slave-3986"
server_key: "e5d827612c8705a1a59d270bea44326e"
server_key_iv: "8b2b533955d3ecb186d9fbe408362329"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.134837 23571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:03.142948 23571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:03.144730 23571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:03.144873 23571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:03.145139 23571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:03.145481 23571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:03.145537 23571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.145577 23571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:03.145608 23571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.151155 23571 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:44621
I20260926 22:45:03.151224 23699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:44621 every 8 connection(s)
I20260926 22:45:03.151536 23571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:45:03.153956 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23571
I20260926 22:45:03.154052 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:45:03.154245 15075 external_mini_cluster.cc:1428] Setting key cff20d4b06ad2f8b8fb70d21c06e1844
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462703, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:03.160320 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.152218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:47529 (local address 127.14.184.254:43753)
0926 22:45:03.152260 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.152265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.152444 (+ 179us) server_negotiation.cc:407] Connection header received
0926 22:45:03.152937 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.152938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.152947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.153002 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.153929 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.154587 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.155786 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.155992 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.156072 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.158047 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:03.158048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:03.158049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:03.158055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:03.159235 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:03.159584 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:03.159584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:03.159585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:03.159631 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:03.159827 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:03.159828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:03.159828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:03.159920 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:03.159969 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:03.160189 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 22:45:03.160248 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:45:03.160460 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.152197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:47529)
0926 22:45:03.152339 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.152377 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.152849 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.153053 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.153056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.153596 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.153823 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.153831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.154728 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.154730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.155716 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.155718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.156082 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.156083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.156207 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.156609 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.156616 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.157966 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:03.159303 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:03.159308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:03.159310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:03.159521 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:03.159685 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:03.159686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:03.159686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:03.159775 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:03.159972 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:03.159976 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:03.160077 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 22:45:03.160155 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:45:03.160263 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260926 22:45:03.160921 23700 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43753
I20260926 22:45:03.161023 23700 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:03.161188 23700 heartbeater.cc:511] Master 127.14.184.254:43753 requested a full tablet report, sending...
I20260926 22:45:03.161545 23241 ts_manager.cc:194] Registered new tserver with Master: 9ecd02fde6914d4dbc1e31d757b66cb5 (127.14.184.195:44621)
I20260926 22:45:03.161901 23241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:47529
I20260926 22:45:03.165961 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab.
Entry for principal oryx/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab.
WARNING: no policy specified for oryx/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab.
Entry for principal oryx/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab.
WARNING: no policy specified for oryx/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab.
Entry for principal oryx/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab.
I20260926 22:45:03.227370 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23303
I20260926 22:45:03.231973 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23437
I20260926 22:45:03.236158 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23571
I20260926 22:45:03.240227 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23209
I20260926 22:45:03.245175 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43753
--webserver_interface=127.14.184.254
--webserver_port=32935
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:03.324234 23709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:03.324431 23709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:03.324453 23709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:03.325583 23709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:45:03.325625 23709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:03.325639 23709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:03.325677 23709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:45:03.325703 23709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:45:03.327171 23709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=32935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:03.327409 23709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:03.327630 23709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:03.332353 23714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.332379 23715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.332379 23717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.332408 23709 server_base.cc:1034] running on GCE node
I20260926 22:45:03.332743 23709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:03.333038 23709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:03.334203 23709 hybrid_clock.cc:648] HybridClock initialized: now 1790462703334184 us; error 30 us; skew 500 ppm
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462703, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:03.336577 23709 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:45:03.336939 23709 webserver.cc:460] Webserver started at http://127.14.184.254:32935/ using document root <none> and password file <none>
I20260926 22:45:03.337114 23709 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:03.337163 23709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:03.338356 23709 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:45:03.338996 23724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.339159 23709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:45:03.339241 23709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "57db547a1baa4b1daa851e19171ceb2a"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "8c70962c70058cd433160a837966c86a"
server_key_iv: "468e46cde80a2ae861f8ce9a266fdc85"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.339550 23709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:03.365906 23709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:03.368176 23709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:03.368332 23709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:03.372041 23709 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43753
I20260926 22:45:03.372092 23776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43753 every 8 connection(s)
I20260926 22:45:03.372452 23709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:45:03.373844 23777 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:45:03.374496 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Bootstrap starting.
I20260926 22:45:03.379469 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23709
I20260926 22:45:03.379755 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:34527
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:43753
--webserver_port=44043
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:45:03.382501 23777 log.cc:824] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:03.384084 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:45:03.384357 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Bootstrap complete.
I20260926 22:45:03.386552 23777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:03.386855 23777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57db547a1baa4b1daa851e19171ceb2a, State: Initialized, Role: FOLLOWER
I20260926 22:45:03.387030 23777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:03.387105 23777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:45:03.387156 23777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:45:03.387204 23777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:45:03.388159 23777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:03.388274 23777 leader_election.cc:302] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57db547a1baa4b1daa851e19171ceb2a; no voters:
I20260926 22:45:03.388469 23777 leader_election.cc:288] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 22:45:03.388502 23781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:45:03.388613 23781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [term 2 LEADER]: Becoming Leader. State: Replica: 57db547a1baa4b1daa851e19171ceb2a, State: Running, Role: LEADER
I20260926 22:45:03.388731 23777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:45:03.388767 23781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } }
I20260926 22:45:03.389135 23783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57db547a1baa4b1daa851e19171ceb2a. Latest consensus state: current_term: 2 leader_uuid: "57db547a1baa4b1daa851e19171ceb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } } }
I20260926 22:45:03.389218 23783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:03.389623 23787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:45:03.390151 23787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:45:03.390388 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "57db547a1baa4b1daa851e19171ceb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57db547a1baa4b1daa851e19171ceb2a" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43753 } } }
I20260926 22:45:03.390493 23787 catalog_manager.cc:1259] Loaded cluster ID: 6bbc5c78172543719dedb2abe4f23a82
I20260926 22:45:03.390586 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:03.390640 23787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:45:03.391476 23787 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:45:03.391661 23787 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 57db547a1baa4b1daa851e19171ceb2a: Loaded TSK: 0
I20260926 22:45:03.391866 23787 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 22:45:03.472932 23779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:03.473158 23779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:03.473198 23779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:03.474422 23779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:03.474483 23779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:03.474526 23779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:45:03.476157 23779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.193.keytab
--principal=oryx/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:34527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=44043
--tserver_master_addrs=127.14.184.254:43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:03.476420 23779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:03.476643 23779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:03.482255 23803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.482313 23806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.482344 23804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.482722 23779 server_base.cc:1034] running on GCE node
I20260926 22:45:03.482920 23779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:03.483150 23779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:03.484306 23779 hybrid_clock.cc:648] HybridClock initialized: now 1790462703484292 us; error 31 us; skew 500 ppm
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462703, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:03.486876 23779 init.cc:377] Logged in from keytab as oryx/127.14.184.193@KRBTEST.COM (short username oryx)
I20260926 22:45:03.487368 23779 webserver.cc:460] Webserver started at http://127.14.184.193:44043/ using document root <none> and password file <none>
I20260926 22:45:03.487609 23779 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:03.487689 23779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:03.489148 23779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:45:03.489831 23813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.490023 23779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:45:03.490109 23779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "aa1f508174224caca4aafe53e510ef97"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "e1c8511635e45e4fb9942fc08fad290e"
server_key_iv: "d9224b695dfac54be7d867ff0c9d3638"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.490445 23779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:03.503093 23779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:03.505110 23779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:03.505256 23779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:03.505486 23779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:03.505772 23779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:03.505831 23779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.505869 23779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:03.505895 23779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.513121 23779 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:34527
I20260926 22:45:03.513191 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:34527 every 8 connection(s)
I20260926 22:45:03.513513 23779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:45:03.514385 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23779
I20260926 22:45:03.514662 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:35191
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:43753
--webserver_port=41925
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.522116 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.514168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:34497)
0926 22:45:03.514385 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.514421 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.515029 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.515639 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.515644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.516315 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.516493 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.516497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.517607 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.517608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.518494 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.518496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.519172 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.519173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.519310 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.519798 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.519813 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.521912 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
W20260926 22:45:03.522238 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.514249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:34497 (local address 127.14.184.254:43753)
0926 22:45:03.515215 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.515223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.515255 (+ 32us) server_negotiation.cc:407] Connection header received
0926 22:45:03.515375 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.515381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.515432 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.515555 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.516616 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.517503 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.518581 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.519069 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.519195 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.522044 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:34497: BlockingRecv error: recv got EOF from 127.14.184.193:34497 (error 108)
Metrics: {"server-negotiator.queue_time_us":916,"thread_start_us":90,"threads_started":1}
W20260926 22:45:03.522650 23927 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43753 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.184.254:43753: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.527083 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.522915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:51041)
0926 22:45:03.523018 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.523029 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.523073 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.523217 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.523218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.523352 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.523393 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.523394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.524019 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.524020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.524538 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.524539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.524892 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.524893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.524955 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.525376 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.525382 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.527001 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260926 22:45:03.527140 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.522966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:51041 (local address 127.14.184.254:43753)
0926 22:45:03.523021 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.523024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.523060 (+ 36us) server_negotiation.cc:407] Connection header received
0926 22:45:03.523130 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.523131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.523142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.523214 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.523467 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.523929 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.524621 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.524820 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.524894 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.527076 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:51041: BlockingRecv error: recv got EOF from 127.14.184.193:51041 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.531816 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.527583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:49247)
0926 22:45:03.527666 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.527676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.527719 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.527866 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.527867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.527991 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.528015 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.528016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.528840 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.528841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.529311 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.529312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.529687 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.529688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.529740 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.530134 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.530139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.531737 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260926 22:45:03.531869 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.527630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:49247 (local address 127.14.184.254:43753)
0926 22:45:03.527664 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.527666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.527705 (+ 39us) server_negotiation.cc:407] Connection header received
0926 22:45:03.527775 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.527776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.527792 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.527864 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.528096 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.528614 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.529409 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.529588 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.529686 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.531810 (+ 2124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:49247: BlockingRecv error: recv got EOF from 127.14.184.193:49247 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 22:45:03.532153 23927 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 22:45:03.600931 23931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:03.601125 23931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:03.601147 23931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:03.602228 23931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:03.602262 23931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:03.602288 23931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:45:03.603634 23931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.194.keytab
--principal=oryx/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:35191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=41925
--tserver_master_addrs=127.14.184.254:43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:03.603818 23931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:03.603981 23931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:03.608865 23937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.608865 23939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.609032 23931 server_base.cc:1034] running on GCE node
W20260926 22:45:03.608884 23936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.609277 23931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:03.609530 23931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:03.610749 23931 hybrid_clock.cc:648] HybridClock initialized: now 1790462703610737 us; error 25 us; skew 500 ppm
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462703, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:03.613186 23931 init.cc:377] Logged in from keytab as oryx/127.14.184.194@KRBTEST.COM (short username oryx)
I20260926 22:45:03.613651 23931 webserver.cc:460] Webserver started at http://127.14.184.194:41925/ using document root <none> and password file <none>
I20260926 22:45:03.613960 23931 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:03.614074 23931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:03.615460 23931 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:45:03.616010 23946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.616174 23931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:03.616276 23931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "dd7da669591d404d8b7e3eac580c9da4"
format_stamp: "Formatted at 2026-09-26 22:45:02 on dist-test-slave-3986"
server_key: "6badbb52c43fd0dcedb31e070dae5b39"
server_key_iv: "78b7084751d8ef1e38a7197f0808a40c"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.616591 23931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:03.631328 23931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:03.632941 23931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:03.633091 23931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:03.633297 23931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:03.633605 23931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:03.633659 23931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.633698 23931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:03.633736 23931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.639642 23931 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:35191
I20260926 22:45:03.639714 24059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:35191 every 8 connection(s)
I20260926 22:45:03.640075 23931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 22:45:03.646781 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.640737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:52957 (local address 127.14.184.254:43753)
0926 22:45:03.640795 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.640797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.640961 (+ 164us) server_negotiation.cc:407] Connection header received
0926 22:45:03.641416 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.641417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.641428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.641476 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.642368 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.642972 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.643922 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.644094 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.644141 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.646736 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:52957: BlockingRecv error: recv got EOF from 127.14.184.194:52957 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:45:03.646801 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.640709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:52957)
0926 22:45:03.640848 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.640885 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.641339 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.641528 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.641531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.642067 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.642281 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.642300 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.643050 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.643051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.643840 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.643842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.644161 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.644161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.644290 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.644685 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.644692 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.646604 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
W20260926 22:45:03.647033 24060 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43753 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.184.254:43753: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
I20260926 22:45:03.649892 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 23931
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.650223 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:44621
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:43753
--webserver_port=37261
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.650936 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.647257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:50419)
0926 22:45:03.647300 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.647310 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.647357 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.647451 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.647451 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.647550 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.647588 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.647588 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.648111 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.648111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.648554 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.648555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.648843 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.648843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.648885 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.649265 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.649269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.650899 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:45:03.651064 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.647321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:50419 (local address 127.14.184.254:43753)
0926 22:45:03.647356 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.647357 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.647366 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:45:03.647393 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.647396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.647407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.647449 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.647654 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.648055 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.648621 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.648782 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.648834 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.650991 (+ 2157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:50419: BlockingRecv error: recv got EOF from 127.14.184.194:50419 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.655474 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.651241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:35673)
0926 22:45:03.651312 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.651320 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.651349 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.651453 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.651454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.651593 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.651622 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.651623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.652159 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.652160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.652925 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.652928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.653245 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.653246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.653311 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.653611 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.653616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.655387 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260926 22:45:03.655488 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.651312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:35673 (local address 127.14.184.254:43753)
0926 22:45:03.651349 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.651351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.651360 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:45:03.651386 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.651386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.651395 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.651448 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.651685 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.652088 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.653012 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.653178 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.653222 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.655454 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:35673: BlockingRecv error: recv got EOF from 127.14.184.194:35673 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 22:45:03.655582 24060 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 22:45:03.728765 24063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:03.728974 24063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:03.728998 24063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:03.730197 24063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:03.730242 24063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:03.730268 24063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:45:03.731712 24063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/krb5kdc/oryx_127.14.184.195.keytab
--principal=oryx/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:44621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=37261
--tserver_master_addrs=127.14.184.254:43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:03.731956 24063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:03.732182 24063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:03.737155 24068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:03.737176 24071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.737193 24063 server_base.cc:1034] running on GCE node
W20260926 22:45:03.737172 24069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:03.737576 24063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:03.737856 24063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:03.738993 24063 hybrid_clock.cc:648] HybridClock initialized: now 1790462703738980 us; error 28 us; skew 500 ppm
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462703, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:03.741477 24063 init.cc:377] Logged in from keytab as oryx/127.14.184.195@KRBTEST.COM (short username oryx)
I20260926 22:45:03.741940 24063 webserver.cc:460] Webserver started at http://127.14.184.195:37261/ using document root <none> and password file <none>
I20260926 22:45:03.742175 24063 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:03.742254 24063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:03.743618 24063 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:45:03.744172 24078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.744341 24063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:03.744429 24063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "9ecd02fde6914d4dbc1e31d757b66cb5"
format_stamp: "Formatted at 2026-09-26 22:45:03 on dist-test-slave-3986"
server_key: "e5d827612c8705a1a59d270bea44326e"
server_key_iv: "8b2b533955d3ecb186d9fbe408362329"
server_key_version: "encryptionkey@0"
I20260926 22:45:03.744807 24063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:03.764683 24063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:03.766789 24063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:03.767005 24063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:03.767267 24063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:03.767568 24063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:03.767623 24063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.767658 24063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:03.767699 24063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:03.773252 24063 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:44621
I20260926 22:45:03.773289 24191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:44621 every 8 connection(s)
I20260926 22:45:03.773602 24063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:45:03.775321 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24063
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 22:45:03.781913 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.774289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:38727 (local address 127.14.184.254:43753)
0926 22:45:03.774380 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.774382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.774530 (+ 148us) server_negotiation.cc:407] Connection header received
0926 22:45:03.775028 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.775029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.775039 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.775090 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.776334 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.777077 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.778190 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.778424 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.778473 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.781864 (+ 3391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:38727: BlockingRecv error: recv got EOF from 127.14.184.195:38727 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 22:45:03.781940 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.774251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:38727)
0926 22:45:03.774416 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.774448 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.774938 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.775139 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.775142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.775888 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.776193 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.776207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.777159 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.777160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.778101 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.778103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.778496 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.778497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.778633 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.779141 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.779148 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.781700 (+ 2552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
W20260926 22:45:03.782276 24192 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43753 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.184.254:43753: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
I20260926 22:45:03.783715 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.775810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.14.184.254:43753)
0926 22:45:03.775924 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.775926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.775936 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:45:03.775992 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.775993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.776004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.776086 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.776887 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.777523 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.778291 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.778499 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.778564 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.779991 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:03.779995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:03.779996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:03.780005 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:03.782062 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:03.782512 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:03.782515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:03.782515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:03.782635 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:03.782927 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:03.782928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:03.782928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:03.783204 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:03.783360 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:03.783489 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 22:45:03.783607 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.786755 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.782518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:59757)
0926 22:45:03.782580 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.782591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.782635 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.782788 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.782789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.782911 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.782952 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.782954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.783538 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.783539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.784336 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.784338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.784684 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.784684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.784745 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.785086 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.785090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.786709 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 22:45:03.786875 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.782545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:59757 (local address 127.14.184.254:43753)
0926 22:45:03.782585 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.782586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.782621 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:45:03.782709 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.782710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.782719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.782786 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.783028 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.783466 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.784428 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.784607 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.784682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.786797 (+ 2115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:59757: BlockingRecv error: recv got EOF from 127.14.184.195:59757 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:03 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:03.791422 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:03.787129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:54129)
0926 22:45:03.787174 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:03.787184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:03.787236 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:03.787351 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:03.787352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:03.787466 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:03.787488 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.787489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.788349 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.788350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.789049 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:03.789051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.789456 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.789457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:03.789508 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.789866 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:03.789883 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:03.791348 (+ 1465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:45:03.791473 23930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:03.787202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:54129 (local address 127.14.184.254:43753)
0926 22:45:03.787255 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:03.787257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:03.787264 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:45:03.787284 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:03.787285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:03.787293 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:03.787347 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:03.787557 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.788256 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.789132 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:03.789378 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:03.789454 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:03.791416 (+ 1962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:54129: BlockingRecv error: recv got EOF from 127.14.184.195:54129 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260926 22:45:03.791654 24192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 22:45:04 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:04 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:04.537119 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:04.532518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:34565)
0926 22:45:04.532655 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:04.532667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:04.532718 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:04.532845 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:04.532846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:04.532998 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:04.533046 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:04.533047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.533663 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.533664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:04.534236 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:04.534237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.534867 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.534867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:04.534908 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:04.535348 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:04.535352 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:04.537054 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260926 22:45:04.537254 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:04.532592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:34565 (local address 127.14.184.254:43753)
0926 22:45:04.532700 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:04.532702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:04.532710 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:04.532765 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:04.532766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:04.532775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:04.532830 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:04.533111 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.533600 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.534307 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.534790 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.534858 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:04.537146 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:34565: BlockingRecv error: recv got EOF from 127.14.184.193:34565 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
Sep 26 22:45:04 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:04 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:04.660820 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:04.655967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:41135)
0926 22:45:04.656134 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:04.656146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:04.656241 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:04.656467 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:04.656472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:04.656655 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:04.656700 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:04.656702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.657362 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.657363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:04.657915 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:04.657917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.658289 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.658290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:04.658348 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:04.658865 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:04.658869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:04.660718 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
W20260926 22:45:04.660854 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:04.656065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:41135 (local address 127.14.184.254:43753)
0926 22:45:04.656138 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:04.656140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:04.656222 (+ 82us) server_negotiation.cc:407] Connection header received
0926 22:45:04.656331 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:04.656332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:04.656351 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:04.656455 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:04.656786 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.657280 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.658002 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.658204 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.658288 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:04.660791 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:41135: BlockingRecv error: recv got EOF from 127.14.184.194:41135 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 22:45:04 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:04 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:04.796528 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:04.792014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:42407)
0926 22:45:04.792143 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:04.792155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:04.792218 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:04.792341 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:04.792341 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:04.792497 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:04.792545 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:04.792546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.793225 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.793226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:04.793786 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:04.793788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.794168 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.794169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:04.794234 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:04.794657 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:04.794661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:04.796416 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260926 22:45:04.796548 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:04.792081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:42407 (local address 127.14.184.254:43753)
0926 22:45:04.792153 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:04.792155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:04.792217 (+ 62us) server_negotiation.cc:407] Connection header received
0926 22:45:04.792259 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:04.792260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:04.792271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:04.792334 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:04.792641 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.793139 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.793884 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:04.794092 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:04.794169 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:04.796488 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:42407: BlockingRecv error: recv got EOF from 127.14.184.195:42407 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 22:45:05 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:05 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:05.541815 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:05.537560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:47843)
0926 22:45:05.537690 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:05.537700 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:05.537756 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:05.537880 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:05.537881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:05.538028 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:05.538077 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:05.538078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.538729 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.538730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:05.539256 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:05.539258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.539583 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.539584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:05.539626 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:05.540045 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:05.540049 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:05.541730 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
W20260926 22:45:05.541878 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:05.537594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:47843 (local address 127.14.184.254:43753)
0926 22:45:05.537696 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:05.537698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:05.537758 (+ 60us) server_negotiation.cc:407] Connection header received
0926 22:45:05.537797 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:05.537798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:05.537809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:05.537866 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:05.538132 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.538616 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.539319 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.539524 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.539571 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:05.541785 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:47843: BlockingRecv error: recv got EOF from 127.14.184.193:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
Sep 26 22:45:05 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:05 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:05.665576 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:05.661282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:35055)
0926 22:45:05.661420 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:05.661431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:05.661492 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:05.661620 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:05.661620 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:05.661776 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:05.661823 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:05.661825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.662444 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.662444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:05.663033 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:05.663035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.663381 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.663382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:05.663426 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:05.663864 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:05.663868 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:05.665521 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260926 22:45:05.665695 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:05.661335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:35055 (local address 127.14.184.254:43753)
0926 22:45:05.661406 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:05.661408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:05.661497 (+ 89us) server_negotiation.cc:407] Connection header received
0926 22:45:05.661538 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:05.661539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:05.661549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:05.661603 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:05.661887 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.662375 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.663118 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.663306 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.663367 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:05.665615 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:35055: BlockingRecv error: recv got EOF from 127.14.184.194:35055 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 22:45:05 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:05 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:05.801445 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:05.797085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:44147)
0926 22:45:05.797257 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:05.797270 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:05.797344 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:05.797468 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:05.797468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:05.797624 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:05.797670 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:05.797672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.798272 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.798273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:05.798820 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:05.798822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.799128 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.799128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:05.799167 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:05.799650 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:05.799655 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:05.801387 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
W20260926 22:45:05.801608 24202 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:05.797232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:44147 (local address 127.14.184.254:43753)
0926 22:45:05.797297 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:05.797299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:05.797324 (+ 25us) server_negotiation.cc:407] Connection header received
0926 22:45:05.797396 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:05.797397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:05.797408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:05.797447 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:05.797732 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.798209 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.798888 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:05.799067 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:05.799118 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:05.801508 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:44147: BlockingRecv error: recv got EOF from 127.14.184.195:44147 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 22:45:06 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:06 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:06.546762 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:06.542310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:47703)
0926 22:45:06.542453 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:06.542469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:06.542516 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:06.542646 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:06.542647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:06.542812 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:06.542866 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:06.542867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.543483 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.543484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:06.544006 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:06.544008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.544318 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.544319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:06.544357 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:06.544846 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:06.544850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:06.546648 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
W20260926 22:45:06.546825 24206 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:06.542372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:47703 (local address 127.14.184.254:43753)
0926 22:45:06.542514 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:06.542521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:06.542549 (+ 28us) server_negotiation.cc:407] Connection header received
0926 22:45:06.542570 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:06.542570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:06.542580 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:06.542628 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:06.542927 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.543420 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.544072 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.544258 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.544305 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:06.546751 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:47703: BlockingRecv error: recv got EOF from 127.14.184.193:47703 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
Sep 26 22:45:06 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:06 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:06.670606 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:06.666049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:38961)
0926 22:45:06.666202 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:06.666212 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:06.666270 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:06.666484 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:06.666484 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:06.666643 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:06.666747 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:06.666748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.667365 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.667366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:06.667917 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:06.667919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.668201 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.668201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:06.668241 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:06.668851 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:06.668856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:06.670553 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260926 22:45:06.670734 24206 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:06.666108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:38961 (local address 127.14.184.254:43753)
0926 22:45:06.666174 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:06.666176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:06.666369 (+ 193us) server_negotiation.cc:407] Connection header received
0926 22:45:06.666400 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:06.666400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:06.666411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:06.666466 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:06.666818 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.667304 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.667983 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.668141 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.668184 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:06.670638 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:38961: BlockingRecv error: recv got EOF from 127.14.184.194:38961 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 22:45:06 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:06 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:06.806562 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:06.801967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:39991)
0926 22:45:06.802143 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:06.802154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:06.802240 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:06.802410 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:06.802411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:06.802595 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:06.802640 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:06.802642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.803319 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.803320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:06.803856 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:06.803857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.804236 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.804237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:06.804300 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:06.804804 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:06.804808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:06.806468 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
W20260926 22:45:06.806600 24206 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:06.802063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:39991 (local address 127.14.184.254:43753)
0926 22:45:06.802136 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:06.802138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:06.802220 (+ 82us) server_negotiation.cc:407] Connection header received
0926 22:45:06.802313 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:06.802314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:06.802328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:06.802404 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:06.802759 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.803237 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.803959 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:06.804153 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:06.804235 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:06.806541 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:39991: BlockingRecv error: recv got EOF from 127.14.184.195:39991 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 22:45:07 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:07 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:07.551599 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:07.547186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:57417)
0926 22:45:07.547299 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:07.547310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:07.547384 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:07.547552 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:07.547553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:07.547705 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:07.547753 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:07.547754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.548439 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.548440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:07.549021 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:07.549023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.549355 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.549356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:07.549394 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:07.549785 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:07.549789 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:07.551542 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
W20260926 22:45:07.551756 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:07.547244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:57417 (local address 127.14.184.254:43753)
0926 22:45:07.547392 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:07.547394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:07.547420 (+ 26us) server_negotiation.cc:407] Connection header received
0926 22:45:07.547450 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:07.547451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:07.547462 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:07.547533 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:07.547814 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.548360 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.549106 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.549287 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.549335 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:07.551628 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:57417: BlockingRecv error: recv got EOF from 127.14.184.193:57417 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
Sep 26 22:45:07 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:07 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:07.680847 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:07.671219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:48217)
0926 22:45:07.671386 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:07.671398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:07.671488 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:07.671662 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:07.671663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:07.671898 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:07.671961 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:07.671964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.672616 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.672617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:07.673135 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:07.673136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.673467 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.673467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:07.673506 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:07.679066 (+ 5560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:07.679072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:07.680780 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
W20260926 22:45:07.680974 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:07.671361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:48217 (local address 127.14.184.254:43753)
0926 22:45:07.671415 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:07.671417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:07.671456 (+ 39us) server_negotiation.cc:407] Connection header received
0926 22:45:07.671574 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:07.671575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:07.671587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:07.671645 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:07.672047 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.672553 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.673208 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.673402 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.673458 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:07.680875 (+ 7417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:48217: BlockingRecv error: recv got EOF from 127.14.184.194:48217 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 26 22:45:07 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:07 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:07.811662 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:07.807094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:58331)
0926 22:45:07.807247 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:07.807257 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:07.807310 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:07.807445 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:07.807446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:07.807599 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:07.807647 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:07.807649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.808289 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.808290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:07.808801 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:07.808803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.809110 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.809111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:07.809150 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:07.809630 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:07.809634 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:07.811571 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
W20260926 22:45:07.811697 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:07.807175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:58331 (local address 127.14.184.254:43753)
0926 22:45:07.807246 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:07.807248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:07.807299 (+ 51us) server_negotiation.cc:407] Connection header received
0926 22:45:07.807372 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:07.807372 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:07.807384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:07.807426 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:07.807712 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.808226 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.808878 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:07.809050 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:07.809099 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:07.811641 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:58331: BlockingRecv error: recv got EOF from 127.14.184.195:58331 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 22:45:08 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:08 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:08.556710 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:08.552046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:41989)
0926 22:45:08.552207 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:08.552217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:08.552262 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:08.552399 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:08.552400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:08.552545 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:08.552593 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:08.552594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.553293 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.553294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:08.554044 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:08.554046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.554389 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.554389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:08.554428 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:08.554920 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:08.554924 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:08.556652 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
W20260926 22:45:08.556864 24214 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:08.552109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:41989 (local address 127.14.184.254:43753)
0926 22:45:08.552284 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:08.552286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:08.552294 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:08.552315 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:08.552316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:08.552325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:08.552387 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:08.552654 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.553196 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.554116 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.554325 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.554380 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:08.556742 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:41989: BlockingRecv error: recv got EOF from 127.14.184.193:41989 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
Sep 26 22:45:08 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:08 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:08.685824 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:08.681412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:36595)
0926 22:45:08.681686 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:08.681700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:08.681770 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:08.681922 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:08.681923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:08.682120 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:08.682176 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:08.682179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.682866 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.682867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:08.683474 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:08.683476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.683768 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.683769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:08.683804 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:08.684199 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:08.684202 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:08.685772 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":65,"threads_started":1}
W20260926 22:45:08.685945 24214 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:08.681662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:36595 (local address 127.14.184.254:43753)
0926 22:45:08.681726 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:08.681727 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:08.681748 (+ 21us) server_negotiation.cc:407] Connection header received
0926 22:45:08.681833 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:08.681834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:08.681845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:08.681905 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:08.682238 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.682756 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.683536 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.683710 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.683760 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:08.685868 (+ 2108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:36595: BlockingRecv error: recv got EOF from 127.14.184.194:36595 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 22:45:08 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:08 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:08.816524 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:08.812125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:44693)
0926 22:45:08.812274 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:08.812283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:08.812335 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:08.812453 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:08.812453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:08.812602 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:08.812651 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:08.812652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.813245 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.813246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:08.813777 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:08.813778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.814110 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.814110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:08.814150 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:08.814644 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:08.814649 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:08.816468 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
W20260926 22:45:08.816658 24214 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:08.812168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:44693 (local address 127.14.184.254:43753)
0926 22:45:08.812230 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:08.812233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:08.812323 (+ 90us) server_negotiation.cc:407] Connection header received
0926 22:45:08.812375 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:08.812376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:08.812387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:08.812437 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:08.812710 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.813184 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.813843 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:08.814047 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:08.814096 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:08.816553 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:44693: BlockingRecv error: recv got EOF from 127.14.184.195:44693 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 22:45:09 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:09 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:09.561563 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:09.557137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:57767)
0926 22:45:09.557311 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:09.557324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:09.557409 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:09.557541 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:09.557541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:09.557699 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:09.557750 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:09.557752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.558365 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.558366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:09.559005 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:09.559008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.559317 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.559317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:09.559357 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:09.559793 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:09.559797 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:09.561469 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
W20260926 22:45:09.561609 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:09.557231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:57767 (local address 127.14.184.254:43753)
0926 22:45:09.557348 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:09.557350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:09.557408 (+ 58us) server_negotiation.cc:407] Connection header received
0926 22:45:09.557453 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:09.557454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:09.557464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:09.557521 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:09.557810 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.558301 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.559072 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.559256 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.559304 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:09.561543 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:57767: BlockingRecv error: recv got EOF from 127.14.184.193:57767 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
Sep 26 22:45:09 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:09 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:09.690953 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:09.686283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:58499)
0926 22:45:09.686460 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:09.686469 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:09.686525 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:09.686647 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:09.686647 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:09.686829 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:09.686885 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:09.686887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.687523 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.687524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:09.688040 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:09.688041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.688415 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.688416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:09.688488 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:09.688946 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:09.688950 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:09.690841 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
W20260926 22:45:09.691020 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:09.686357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:58499 (local address 127.14.184.254:43753)
0926 22:45:09.686425 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:09.686427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:09.686512 (+ 85us) server_negotiation.cc:407] Connection header received
0926 22:45:09.686583 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:09.686584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:09.686594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:09.686635 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:09.686946 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.687456 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.688164 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.688353 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.688404 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:09.690940 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:58499: BlockingRecv error: recv got EOF from 127.14.184.194:58499 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 22:45:09 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:09 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:09.821290 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:09.816961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:60181)
0926 22:45:09.817100 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:09.817110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:09.817156 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:09.817283 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:09.817283 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:09.817442 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:09.817489 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:09.817491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.818096 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.818097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:09.818632 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:09.818633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.819011 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.819012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:09.819056 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:09.819542 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:09.819546 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:09.821232 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
W20260926 22:45:09.821425 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:09.817072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:60181 (local address 127.14.184.254:43753)
0926 22:45:09.817146 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:09.817148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:09.817157 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:45:09.817213 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:09.817213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:09.817225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:09.817266 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:09.817550 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.818033 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.818770 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:09.818947 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:09.819000 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:09.821321 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:60181: BlockingRecv error: recv got EOF from 127.14.184.195:60181 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 22:45:10 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:10 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:10.566432 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:10.562041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:35787)
0926 22:45:10.562172 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:10.562183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:10.562229 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:10.562369 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:10.562370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:10.562518 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:10.562566 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:10.562568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.563229 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.563230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:10.563749 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:10.563750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.564064 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.564065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:10.564106 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:10.564524 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:10.564528 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:10.566322 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
W20260926 22:45:10.566476 24222 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:10.562125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:35787 (local address 127.14.184.254:43753)
0926 22:45:10.562249 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:10.562251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:10.562258 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:45:10.562278 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:10.562278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:10.562288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:10.562358 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:10.562629 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.563165 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.563818 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.564002 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.564050 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:10.566407 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:35787: BlockingRecv error: recv got EOF from 127.14.184.193:35787 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
Sep 26 22:45:10 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:10 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:10.695964 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:10.691431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:45049)
0926 22:45:10.691559 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:10.691571 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:10.691633 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:10.691752 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:10.691752 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:10.691912 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:10.691963 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:10.691965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.692571 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.692571 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:10.693154 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:10.693156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.693463 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.693464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:10.693502 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:10.693987 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:10.693991 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:10.695907 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
W20260926 22:45:10.696071 24222 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:10.691479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:45049 (local address 127.14.184.254:43753)
0926 22:45:10.691542 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:10.691544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:10.691632 (+ 88us) server_negotiation.cc:407] Connection header received
0926 22:45:10.691676 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:10.691677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:10.691687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:10.691738 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:10.692022 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.692507 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.693215 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.693403 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.693456 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:10.695998 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:45049: BlockingRecv error: recv got EOF from 127.14.184.194:45049 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 22:45:10 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:10 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:10.826290 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:10.821699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:36471)
0926 22:45:10.821854 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:10.821866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:10.821954 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:10.822100 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:10.822101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:10.822274 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:10.822321 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:10.822322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.823062 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.823063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:10.823596 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:10.823597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.823961 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.823962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:10.824025 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:10.824531 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:10.824536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:10.826198 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
W20260926 22:45:10.826328 24222 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:10.821773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:36471 (local address 127.14.184.254:43753)
0926 22:45:10.821851 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:10.821853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:10.821936 (+ 83us) server_negotiation.cc:407] Connection header received
0926 22:45:10.822010 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:10.822011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:10.822031 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:10.822095 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:10.822408 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.822975 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.823682 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:10.823879 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:10.823952 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:10.826269 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:36471: BlockingRecv error: recv got EOF from 127.14.184.195:36471 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 22:45:11 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:11 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:11.571427 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:11.567041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:52459)
0926 22:45:11.567181 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:11.567192 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:11.567243 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:11.567424 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:11.567424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:11.567574 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:11.567623 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:11.567625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.568253 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.568254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:11.568764 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:11.568765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.569074 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.569075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:11.569114 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:11.569533 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:11.569537 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:11.571327 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260926 22:45:11.571599 24226 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:11.567165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:52459 (local address 127.14.184.254:43753)
0926 22:45:11.567285 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:11.567287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:11.567295 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:11.567326 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:11.567327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:11.567337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:11.567405 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:11.567697 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.568186 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.568833 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.569013 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.569062 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:11.571467 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:52459: BlockingRecv error: recv got EOF from 127.14.184.193:52459 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
Sep 26 22:45:11 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:11 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:11.701143 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:11.696512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:51795)
0926 22:45:11.696665 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:11.696678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:11.696739 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:11.696878 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:11.696878 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:11.697035 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:11.697082 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:11.697083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.697827 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.697828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:11.698345 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:11.698346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.698656 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.698657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:11.698749 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:11.699250 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:11.699254 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:11.701085 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
W20260926 22:45:11.701265 24226 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:11.696595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:51795 (local address 127.14.184.254:43753)
0926 22:45:11.696656 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:11.696657 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:11.696737 (+ 80us) server_negotiation.cc:407] Connection header received
0926 22:45:11.696796 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:11.696796 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:11.696807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:11.696859 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:11.697142 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.697763 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.698412 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.698595 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.698644 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:11.701169 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:51795: BlockingRecv error: recv got EOF from 127.14.184.194:51795 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 22:45:11 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:11 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:11.831357 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:11.826963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:41167)
0926 22:45:11.827112 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:11.827122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:11.827182 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:11.827313 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:11.827313 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:11.827464 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:11.827512 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:11.827514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.828114 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.828115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:11.828613 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:11.828614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.828980 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.828981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:11.829050 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:11.829534 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:11.829540 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:11.831254 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
W20260926 22:45:11.831398 24226 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:11.827045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41167 (local address 127.14.184.254:43753)
0926 22:45:11.827137 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:11.827140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:11.827168 (+ 28us) server_negotiation.cc:407] Connection header received
0926 22:45:11.827235 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:11.827236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:11.827248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:11.827296 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:11.827575 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.828052 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.828676 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:11.828908 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:11.828980 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:11.831328 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:41167: BlockingRecv error: recv got EOF from 127.14.184.195:41167 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 22:45:12 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:12 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:12.576090 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:12.571831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:40791)
0926 22:45:12.571954 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:12.571967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:12.572025 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:12.572138 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:12.572138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:12.572283 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:12.572330 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:12.572331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.572967 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.572968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:12.573476 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:12.573477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.573796 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.573796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:12.573855 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:12.574264 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:12.574268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:12.576032 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
W20260926 22:45:12.576193 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:12.571909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40791 (local address 127.14.184.254:43753)
0926 22:45:12.572038 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:12.572040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:12.572047 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:45:12.572072 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:12.572072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:12.572081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:12.572133 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:12.572389 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.572903 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.573551 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.573735 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.573783 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:12.576118 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:40791: BlockingRecv error: recv got EOF from 127.14.184.193:40791 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
Sep 26 22:45:12 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:12 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:12.710317 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:12.701543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:41575)
0926 22:45:12.701717 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:12.701726 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:12.701779 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:12.701922 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:12.701922 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:12.702079 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:12.702128 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:12.702129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.702785 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.702786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:12.703290 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:12.703292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.703591 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.703592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:12.703637 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:12.708486 (+ 4849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:12.708491 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:12.710251 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
W20260926 22:45:12.710405 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:12.701627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:41575 (local address 127.14.184.254:43753)
0926 22:45:12.701679 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:12.701681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:12.701777 (+ 96us) server_negotiation.cc:407] Connection header received
0926 22:45:12.701822 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:12.701823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:12.701833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:12.701906 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:12.702184 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.702700 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.703352 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.703531 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.703579 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:12.710358 (+ 6779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:41575: BlockingRecv error: recv got EOF from 127.14.184.194:41575 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 26 22:45:12 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:12 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:12.836493 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:12.831882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:57889)
0926 22:45:12.832010 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:12.832021 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:12.832097 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:12.832256 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:12.832257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:12.832449 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:12.832495 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:12.832497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.833166 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.833167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:12.833685 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:12.833687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.834054 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.834055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:12.834111 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:12.834634 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:12.834638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:12.836434 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
W20260926 22:45:12.836583 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:12.832060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:57889 (local address 127.14.184.254:43753)
0926 22:45:12.832133 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:12.832135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:12.832143 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:12.832169 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:12.832170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:12.832185 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:12.832250 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:12.832576 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.833079 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.833768 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:12.833959 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:12.834052 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:12.836521 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:57889: BlockingRecv error: recv got EOF from 127.14.184.195:57889 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 22:45:13 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:13 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:13.581125 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:13.576691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:60549)
0926 22:45:13.576872 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:13.576889 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:13.576945 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:13.577127 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:13.577128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:13.577285 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:13.577335 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:13.577336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.577960 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.577960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:13.578471 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:13.578472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.578796 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.578796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:13.578839 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:13.579246 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:13.579250 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:13.581059 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":60,"threads_started":1}
W20260926 22:45:13.581247 24234 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:13.576889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:60549 (local address 127.14.184.254:43753)
0926 22:45:13.577003 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:13.577004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:13.577012 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:13.577034 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:13.577034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:13.577044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:13.577104 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:13.577399 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.577898 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.578535 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.578738 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.578785 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:13.581158 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:60549: BlockingRecv error: recv got EOF from 127.14.184.193:60549 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
Sep 26 22:45:13 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:13 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:13.715093 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:13.710827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:45051)
0926 22:45:13.710948 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:13.710959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:13.711016 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:13.711142 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:13.711143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:13.711297 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:13.711343 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:13.711344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.711956 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.711957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:13.712474 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:13.712476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.712788 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.712789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:13.712827 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:13.713253 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:13.713257 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:13.714993 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260926 22:45:13.715147 24234 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:13.710903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:45051 (local address 127.14.184.254:43753)
0926 22:45:13.710983 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:13.710988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:13.711000 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:45:13.711071 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:13.711071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:13.711081 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:13.711118 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:13.711414 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.711894 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.712546 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.712726 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.712776 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:13.715081 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:45051: BlockingRecv error: recv got EOF from 127.14.184.194:45051 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 22:45:13 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:13 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:13.841428 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:13.836998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:51803)
0926 22:45:13.837153 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:13.837167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:13.837233 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:13.837368 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:13.837368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:13.837522 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:13.837568 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:13.837569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.838167 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.838168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:13.838709 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:13.838711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.839023 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.839024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:13.839063 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:13.839620 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:13.839624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:13.841365 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
W20260926 22:45:13.841560 24234 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:13.837141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:51803 (local address 127.14.184.254:43753)
0926 22:45:13.837206 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:13.837208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:13.837219 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:45:13.837293 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:13.837294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:13.837306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:13.837359 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:13.837626 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.838104 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.838781 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:13.838960 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:13.839012 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:13.841460 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:51803: BlockingRecv error: recv got EOF from 127.14.184.195:51803 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 22:45:14 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:14 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:14.585933 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:14.581625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:33537)
0926 22:45:14.581781 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:14.581791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:14.581843 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:14.581977 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:14.581978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:14.582139 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:14.582183 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:14.582185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.582816 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.582817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:14.583358 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:14.583359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.583667 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.583668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:14.583705 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:14.584125 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:14.584129 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:14.585868 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
W20260926 22:45:14.586094 24238 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:14.581721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:33537 (local address 127.14.184.254:43753)
0926 22:45:14.581829 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:14.581831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:14.581839 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:14.581895 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:14.581896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:14.581906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:14.581960 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:14.582249 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.582755 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.583426 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.583609 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.583656 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:14.585989 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:33537: BlockingRecv error: recv got EOF from 127.14.184.193:33537 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
Sep 26 22:45:14 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:14 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:14.720417 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:14.715595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:55333)
0926 22:45:14.715772 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:14.715784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:14.715858 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:14.715991 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:14.715991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:14.716181 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:14.716225 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:14.716227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.716889 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.716890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:14.717444 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:14.717445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.717896 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.717896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:14.717954 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:14.718463 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:14.718468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:14.720351 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
W20260926 22:45:14.720525 24238 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:14.715681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:55333 (local address 127.14.184.254:43753)
0926 22:45:14.715755 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:14.715757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:14.715850 (+ 93us) server_negotiation.cc:407] Connection header received
0926 22:45:14.715921 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:14.715922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:14.715934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:14.715986 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:14.716305 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.716799 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.717530 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.717830 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.717898 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:14.720435 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:55333: BlockingRecv error: recv got EOF from 127.14.184.194:55333 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 22:45:14 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:14 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:14.846736 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:14.841899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:60203)
0926 22:45:14.842066 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:14.842077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:14.842170 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:14.842341 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:14.842342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:14.842528 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:14.842571 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:14.842573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.843296 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.843297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:14.843841 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:14.843843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.844247 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.844247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:14.844334 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:14.844846 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:14.844858 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:14.846602 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
W20260926 22:45:14.846765 24238 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:14.841977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:60203 (local address 127.14.184.254:43753)
0926 22:45:14.842047 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:14.842050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:14.842147 (+ 97us) server_negotiation.cc:407] Connection header received
0926 22:45:14.842248 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:14.842249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:14.842264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:14.842331 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:14.842650 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.843217 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.843959 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:14.844165 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:14.844237 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:14.846702 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:60203: BlockingRecv error: recv got EOF from 127.14.184.195:60203 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 22:45:15 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:15 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:15.591254 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:15.586395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:42931)
0926 22:45:15.586660 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:15.586686 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:15.586754 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:15.586899 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:15.586900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:15.587063 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:15.587110 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:15.587111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.587747 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.587748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:15.588282 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:15.588283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.588630 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.588631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:15.588671 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:15.589274 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:15.589278 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:15.591185 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":185,"threads_started":1}
W20260926 22:45:15.591364 24242 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:15.586575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:42931 (local address 127.14.184.254:43753)
0926 22:45:15.586710 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:15.586712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:15.586737 (+ 25us) server_negotiation.cc:407] Connection header received
0926 22:45:15.586809 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:15.586809 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:15.586818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:15.586881 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:15.587168 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.587676 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.588360 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.588565 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.588619 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:15.591269 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:42931: BlockingRecv error: recv got EOF from 127.14.184.193:42931 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":50,"threads_started":1}
Sep 26 22:45:15 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:15 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:15.725433 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:15.720825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:36163)
0926 22:45:15.720955 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:15.720969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:15.721034 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:15.721222 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:15.721223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:15.721404 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:15.721450 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:15.721452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.722106 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.722107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:15.722633 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:15.722634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.723068 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.723069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:15.723129 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:15.723570 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:15.723574 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:15.725341 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
W20260926 22:45:15.725486 24242 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:15.720931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:36163 (local address 127.14.184.254:43753)
0926 22:45:15.720998 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:15.721000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:15.721013 (+ 13us) server_negotiation.cc:407] Connection header received
0926 22:45:15.721125 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:15.721126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:15.721139 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:15.721200 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:15.721530 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.722021 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.722786 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.722979 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.723066 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:15.725424 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:36163: BlockingRecv error: recv got EOF from 127.14.184.194:36163 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 22:45:15 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:15 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:15.851859 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:15.847240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:57491)
0926 22:45:15.847406 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:15.847417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:15.847487 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:15.847615 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:15.847615 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:15.847811 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:15.847863 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:15.847865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.848530 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.848531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:15.849075 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:15.849077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.849448 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.849449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:15.849512 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:15.849944 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:15.849948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:15.851762 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
W20260926 22:45:15.851892 24242 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:15.847313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:57491 (local address 127.14.184.254:43753)
0926 22:45:15.847403 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:15.847405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:15.847467 (+ 62us) server_negotiation.cc:407] Connection header received
0926 22:45:15.847541 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:15.847542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:15.847555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:15.847599 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:15.847943 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.848446 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.849162 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:15.849363 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:15.849448 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:15.851836 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:57491: BlockingRecv error: recv got EOF from 127.14.184.195:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 22:45:16 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:16 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:16.596414 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:16.591679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:59719)
0926 22:45:16.591837 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:16.591847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:16.591906 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:16.592049 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:16.592050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:16.592210 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:16.592256 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:16.592258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.593052 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.593053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:16.593657 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:16.593659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.594072 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.594073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:16.594135 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:16.594735 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:16.594747 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:16.596348 (+ 1601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
W20260926 22:45:16.596485 24246 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:16.591758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59719 (local address 127.14.184.254:43753)
0926 22:45:16.591855 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:16.591857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:16.591890 (+ 33us) server_negotiation.cc:407] Connection header received
0926 22:45:16.591963 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:16.591964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:16.591974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:16.592034 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:16.592372 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.592983 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.593763 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.593991 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.594062 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:16.596434 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:59719: BlockingRecv error: recv got EOF from 127.14.184.193:59719 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":46,"threads_started":1}
Sep 26 22:45:16 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:16 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:16.730298 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:16.725885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:55023)
0926 22:45:16.726016 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:16.726030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:16.726079 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:16.726213 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:16.726214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:16.726393 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:16.726442 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:16.726444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.727090 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.727091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:16.727619 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:16.727621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.727964 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.727965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:16.728006 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:16.728498 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:16.728503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:16.730241 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
W20260926 22:45:16.730412 24246 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:16.725996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:55023 (local address 127.14.184.254:43753)
0926 22:45:16.726061 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:16.726062 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:16.726069 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:45:16.726127 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:16.726127 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:16.726137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:16.726182 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:16.726505 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.727015 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.727705 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.727895 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.727949 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:16.730336 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:55023: BlockingRecv error: recv got EOF from 127.14.184.194:55023 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 22:45:16 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:16 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:16.856895 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:16.852348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:47215)
0926 22:45:16.852524 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:16.852535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:16.852607 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:16.852779 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:16.852781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:16.853050 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:16.853124 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:16.853127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.853821 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.853821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:16.854337 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:16.854339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.854632 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.854633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:16.854775 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:16.855204 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:16.855209 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:16.856831 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
W20260926 22:45:16.857012 24246 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:16.852420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:47215 (local address 127.14.184.254:43753)
0926 22:45:16.852508 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:16.852511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:16.852588 (+ 77us) server_negotiation.cc:407] Connection header received
0926 22:45:16.852664 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:16.852664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:16.852678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:16.852736 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:16.853213 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.853754 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.854406 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:16.854577 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:16.854623 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:16.856917 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:47215: BlockingRecv error: recv got EOF from 127.14.184.195:47215 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 22:45:17 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:17 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:17.601619 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:17.596914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:59369)
0926 22:45:17.597080 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:17.597114 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:17.597174 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:17.597360 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:17.597362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:17.597623 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:17.597681 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:17.597683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.598371 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.598372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:17.598939 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:17.598941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.599309 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.599310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:17.599370 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:17.599783 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:17.599788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:17.601522 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
W20260926 22:45:17.601680 24250 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:17.597051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59369 (local address 127.14.184.254:43753)
0926 22:45:17.597175 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:17.597177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:17.597186 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:45:17.597222 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:17.597223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:17.597236 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:17.597352 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:17.597782 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.598285 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.599025 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.599224 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.599307 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:17.601606 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:59369: BlockingRecv error: recv got EOF from 127.14.184.193:59369 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":54,"threads_started":1}
Sep 26 22:45:17 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:17 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:17.740526 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:17.730771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:60371)
0926 22:45:17.730932 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:17.730945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:17.731028 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:17.731164 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:17.731165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:17.731353 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:17.731398 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:17.731400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.732078 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.732079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:17.732606 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:17.732607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.732982 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.732983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:17.733048 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:17.738649 (+ 5601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:17.738654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:17.740435 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
W20260926 22:45:17.740664 24250 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:17.730858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:60371 (local address 127.14.184.254:43753)
0926 22:45:17.730921 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:17.730924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:17.731008 (+ 84us) server_negotiation.cc:407] Connection header received
0926 22:45:17.731092 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:17.731093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:17.731105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:17.731151 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:17.731487 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.731996 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.732701 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.732900 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.732982 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:17.740552 (+ 7570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:60371: BlockingRecv error: recv got EOF from 127.14.184.194:60371 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 22:45:17 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:17 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.195@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:17.861776 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:17.857337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.195:60771)
0926 22:45:17.857540 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:17.857550 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:17.857612 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:17.857732 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:17.857732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:17.857899 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:17.857947 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:17.857948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.858567 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.858568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:17.859119 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:17.859121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.859498 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.859499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:17.859538 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:17.860034 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:17.860039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:17.861722 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":139,"threads_started":1}
W20260926 22:45:17.861954 24250 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:17.857418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:60771 (local address 127.14.184.254:43753)
0926 22:45:17.857493 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:17.857495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:17.857600 (+ 105us) server_negotiation.cc:407] Connection header received
0926 22:45:17.857655 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:17.857656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:17.857667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:17.857717 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:17.858015 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.858505 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.859214 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:17.859427 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:17.859492 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:17.861870 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.195:60771: BlockingRecv error: recv got EOF from 127.14.184.195:60771 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 22:45:18 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:18 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.193@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:18.606981 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:18.602162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.193:60891)
0926 22:45:18.602307 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:18.602344 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:18.602396 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:18.602536 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:18.602536 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:18.602737 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:18.602787 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:18.602789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.603429 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.603430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:18.604182 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:18.604185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.604574 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.604575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:18.604615 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:18.605056 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:18.605060 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:18.606925 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
W20260926 22:45:18.607093 24254 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:18.602229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:60891 (local address 127.14.184.254:43753)
0926 22:45:18.602390 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:18.602392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:18.602401 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:45:18.602455 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:18.602456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:18.602465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:18.602515 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:18.602859 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.603354 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.604271 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.604514 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.604565 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:18.607012 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.193:60891: BlockingRecv error: recv got EOF from 127.14.184.193:60891 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
Sep 26 22:45:18 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 22:45:18 dist-test-slave-3986 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.184.194@KRBTEST.COM for oryx/127.14.184.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 22:45:18.745895 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:18.741030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43753 (local address 127.14.184.194:45705)
0926 22:45:18.741212 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:18.741226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:18.741306 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:18.741490 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:18.741491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:18.741664 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:18.741709 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:18.741711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.742417 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.742418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:18.743030 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:18.743032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.743448 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.743449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:18.743515 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:18.744086 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:18.744091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:18.745839 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.184.254:43753: Server oryx/127.14.184.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
W20260926 22:45:18.746047 24254 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:18.741135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:45705 (local address 127.14.184.254:43753)
0926 22:45:18.741216 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:18.741219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:18.741285 (+ 66us) server_negotiation.cc:407] Connection header received
0926 22:45:18.741387 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:18.741388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:18.741400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:18.741483 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:18.741798 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.742337 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.743115 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.743347 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.743449 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:18.745968 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.184.194:45705: BlockingRecv error: recv got EOF from 127.14.184.194:45705 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 22:45:18.775866 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23779
I20260926 22:45:18.780272 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23931
I20260926 22:45:18.784740 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24063
I20260926 22:45:18.789049 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 23709
2026-09-26T22:45:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17473 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T22:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:45:18Z Disabled control of system clock
I20260926 22:45:18.816818 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37161
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:37161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 22:45:18.895740 24259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:18.895944 24259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:18.895964 24259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:18.897042 24259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:45:18.897075 24259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:18.897089 24259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:18.897100 24259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:45:18.897112 24259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:45:18.898361 24259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:37161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:37161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:18.898545 24259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:18.898795 24259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:18.903640 24265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:18.903683 24259 server_base.cc:1034] running on GCE node
W20260926 22:45:18.903659 24264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:18.903650 24267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:18.904114 24259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:18.904405 24259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:18.905542 24259 hybrid_clock.cc:648] HybridClock initialized: now 1790462718905530 us; error 26 us; skew 500 ppm
I20260926 22:45:18.906762 24259 webserver.cc:460] Webserver started at http://127.14.184.254:45437/ using document root <none> and password file <none>
I20260926 22:45:18.907051 24259 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:18.907097 24259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:18.907263 24259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:18.908336 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "36d1bf2f5a724f85a09d5895a4d7abf7"
format_stamp: "Formatted at 2026-09-26 22:45:18 on dist-test-slave-3986"
server_key: "d8af429b7a1ba347a364c9093f9d90f0"
server_key_iv: "4a504a2bfc820cfea0b41a5c12225eea"
server_key_version: "encryptionkey@0"
I20260926 22:45:18.908692 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "36d1bf2f5a724f85a09d5895a4d7abf7"
format_stamp: "Formatted at 2026-09-26 22:45:18 on dist-test-slave-3986"
server_key: "d8af429b7a1ba347a364c9093f9d90f0"
server_key_iv: "4a504a2bfc820cfea0b41a5c12225eea"
server_key_version: "encryptionkey@0"
I20260926 22:45:18.910163 24259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:45:18.910861 24273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:18.911026 24259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:18.911113 24259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "36d1bf2f5a724f85a09d5895a4d7abf7"
format_stamp: "Formatted at 2026-09-26 22:45:18 on dist-test-slave-3986"
server_key: "d8af429b7a1ba347a364c9093f9d90f0"
server_key_iv: "4a504a2bfc820cfea0b41a5c12225eea"
server_key_version: "encryptionkey@0"
I20260926 22:45:18.911182 24259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:18.917577 24259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:18.917831 24259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:18.917971 24259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:18.921825 24259 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:37161
I20260926 22:45:18.921860 24325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:37161 every 8 connection(s)
I20260926 22:45:18.922171 24259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:45:18.922817 24326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:18.925179 24326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7: Bootstrap starting.
I20260926 22:45:18.925731 24326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:18.926021 24326 log.cc:824] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:18.926659 24326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7: No bootstrap required, opened a new log
I20260926 22:45:18.928241 24326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37161 } }
I20260926 22:45:18.928366 24326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:18.928390 24326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d1bf2f5a724f85a09d5895a4d7abf7, State: Initialized, Role: FOLLOWER
I20260926 22:45:18.928552 24326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37161 } }
I20260926 22:45:18.928610 24326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:45:18.928630 24326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:45:18.928696 24326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:18.929184 24326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37161 } }
I20260926 22:45:18.929291 24326 leader_election.cc:302] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36d1bf2f5a724f85a09d5895a4d7abf7; no voters:
I20260926 22:45:18.929466 24326 leader_election.cc:288] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:45:18.929512 24329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:45:18.929711 24329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [term 1 LEADER]: Becoming Leader. State: Replica: 36d1bf2f5a724f85a09d5895a4d7abf7, State: Running, Role: LEADER
I20260926 22:45:18.929714 24326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:45:18.929831 24329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37161 } }
I20260926 22:45:18.930198 24331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d1bf2f5a724f85a09d5895a4d7abf7. Latest consensus state: current_term: 1 leader_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37161 } } }
I20260926 22:45:18.930320 24330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d1bf2f5a724f85a09d5895a4d7abf7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 37161 } } }
I20260926 22:45:18.930362 24331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:18.930379 24330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7 [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:18.930723 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24259
I20260926 22:45:18.930843 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:45:18.930922 24338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:45:18.931053 15075 external_mini_cluster.cc:1428] Setting key f28568b15031896d894ee32315b7bada
I20260926 22:45:18.931434 24338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:45:18.933882 24338 catalog_manager.cc:1347] Generated new cluster ID: f3a56e4fd2734f60ab1f1f100a3a5b42
I20260926 22:45:18.933936 24338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:45:18.936681 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:18.931463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.14.184.254:37161)
0926 22:45:18.931998 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:18.932005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:18.932037 (+ 32us) server_negotiation.cc:407] Connection header received
0926 22:45:18.932166 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:18.932172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:18.932238 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:18.932379 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:18.932586 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.933442 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.934532 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:18.935200 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:18.935342 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:18.935975 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:18.935979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:18.935980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:18.936178 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:18.936219 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:18.936233 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:18.936305 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:18.936373 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 22:45:18.936453 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":77,"threads_started":1}
I20260926 22:45:18.942329 24338 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:45:18.943011 24338 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:45:18.954985 24338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36d1bf2f5a724f85a09d5895a4d7abf7: Generated new TSK 0
I20260926 22:45:18.955179 24338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:45:18.957010 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37161
--builtin_ntp_servers=127.14.184.212:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 22:45:19.035171 24350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.035357 24350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.035378 24350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.036582 24350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.036620 24350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.036644 24350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:45:19.037926 24350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.038137 24350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.038431 24350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.044188 24355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.044219 24356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.044211 24358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.044378 24350 server_base.cc:1034] running on GCE node
I20260926 22:45:19.044579 24350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.044819 24350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.045974 24350 hybrid_clock.cc:648] HybridClock initialized: now 1790462719045960 us; error 29 us; skew 500 ppm
I20260926 22:45:19.047297 24350 webserver.cc:460] Webserver started at http://127.14.184.193:42849/ using document root <none> and password file <none>
I20260926 22:45:19.047538 24350 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.047605 24350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.047739 24350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.048761 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "920eb538028a4505a3efe9e599cb851e"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "262d29be29feef7b7bf38544137f0f13"
server_key_iv: "fa84be89920aac225a7f5eee624ed3f9"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.049116 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "920eb538028a4505a3efe9e599cb851e"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "262d29be29feef7b7bf38544137f0f13"
server_key_iv: "fa84be89920aac225a7f5eee624ed3f9"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.050411 24350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:45:19.051262 24364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.051455 24350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 22:45:19.051540 24350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "920eb538028a4505a3efe9e599cb851e"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "262d29be29feef7b7bf38544137f0f13"
server_key_iv: "fa84be89920aac225a7f5eee624ed3f9"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.051628 24350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.071648 24350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.071959 24350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.072113 24350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.072350 24350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:19.072670 24350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:19.072726 24350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.072759 24350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:19.072788 24350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.078518 24350 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:44433
I20260926 22:45:19.078557 24477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:44433 every 8 connection(s)
I20260926 22:45:19.078924 24350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:45:19.081945 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24350
I20260926 22:45:19.082046 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:45:19.082245 15075 external_mini_cluster.cc:1428] Setting key 0c07039403d4c55151d9af6e39552539
I20260926 22:45:19.083307 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37161
--builtin_ntp_servers=127.14.184.212:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 22:45:19.084560 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.079655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:60225 (local address 127.14.184.254:37161)
0926 22:45:19.079704 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.079707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.080009 (+ 302us) server_negotiation.cc:407] Connection header received
0926 22:45:19.080524 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.080528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.080541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.080597 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.081201 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.081702 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.083134 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.083325 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.083417 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:19.084294 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.084295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.084296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.084299 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.084316 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.084320 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.084364 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.084464 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.084486 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:45:19.084841 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.079672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37161 (local address 127.14.184.193:60225)
0926 22:45:19.079868 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:19.079901 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:19.080416 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:19.080658 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:19.080661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:19.080670 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:19.081127 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:19.081131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.081826 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.081827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:19.083053 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:19.083056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.083443 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.083444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:19.083608 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:19.084141 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:45:19.084141 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:19.084180 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:45:19.084208 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:19.084383 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:19.084388 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:19.084391 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:45:19.084460 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 22:45:19.084577 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260926 22:45:19.085326 24478 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37161
I20260926 22:45:19.085434 24478 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:19.085629 24478 heartbeater.cc:511] Master 127.14.184.254:37161 requested a full tablet report, sending...
I20260926 22:45:19.086115 24290 ts_manager.cc:194] Registered new tserver with Master: 920eb538028a4505a3efe9e599cb851e (127.14.184.193:44433)
I20260926 22:45:19.086796 24290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.193:60225
W20260926 22:45:19.163995 24481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.164175 24481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.164194 24481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.165346 24481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.165385 24481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.165408 24481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:45:19.166731 24481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.166957 24481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.167232 24481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.172160 24489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.172163 24487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.172163 24486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.172253 24481 server_base.cc:1034] running on GCE node
I20260926 22:45:19.172606 24481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.172847 24481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.174005 24481 hybrid_clock.cc:648] HybridClock initialized: now 1790462719173999 us; error 36 us; skew 500 ppm
I20260926 22:45:19.175350 24481 webserver.cc:460] Webserver started at http://127.14.184.194:40023/ using document root <none> and password file <none>
I20260926 22:45:19.175593 24481 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.175664 24481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.175797 24481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.176828 24481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "767f5488a05d4ac0b84db0c10f3afe7e"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "6de1665b7cd2c6e936834f4b476c0c6e"
server_key_iv: "37fe6d641f31168d2f0854a752d5b2f6"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.177189 24481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "767f5488a05d4ac0b84db0c10f3afe7e"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "6de1665b7cd2c6e936834f4b476c0c6e"
server_key_iv: "37fe6d641f31168d2f0854a752d5b2f6"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.178489 24481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:45:19.179332 24495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.179512 24481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:45:19.179608 24481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "767f5488a05d4ac0b84db0c10f3afe7e"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "6de1665b7cd2c6e936834f4b476c0c6e"
server_key_iv: "37fe6d641f31168d2f0854a752d5b2f6"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.179697 24481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.191531 24481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.191817 24481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.191954 24481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.192191 24481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:19.192512 24481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:19.192567 24481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.192606 24481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:19.192634 24481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.198354 24481 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:43067
I20260926 22:45:19.198390 24608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:43067 every 8 connection(s)
I20260926 22:45:19.198828 24481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:45:19.203938 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.199574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:33227 (local address 127.14.184.254:37161)
0926 22:45:19.199624 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.199626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.199906 (+ 280us) server_negotiation.cc:407] Connection header received
0926 22:45:19.200372 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.200373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.200385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.200430 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.201052 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.201584 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.202525 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.202789 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.202843 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:19.203724 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.203725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.203725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.203729 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.203743 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.203747 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.203796 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.203860 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.203881 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:45:19.204162 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.199533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37161 (local address 127.14.184.194:33227)
0926 22:45:19.199722 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:19.199765 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:19.200267 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:19.200483 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:19.200486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:19.200494 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:19.200963 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:19.200967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.201674 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.201675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:19.202442 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:19.202444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.202880 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.202881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:19.202998 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:19.203584 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:45:19.203585 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:19.203615 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:45:19.203638 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:19.203796 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:19.203798 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:19.203801 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:45:19.203847 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 22:45:19.203921 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260926 22:45:19.204710 24609 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37161
I20260926 22:45:19.204826 24609 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:19.204967 24609 heartbeater.cc:511] Master 127.14.184.254:37161 requested a full tablet report, sending...
I20260926 22:45:19.205322 24290 ts_manager.cc:194] Registered new tserver with Master: 767f5488a05d4ac0b84db0c10f3afe7e (127.14.184.194:43067)
I20260926 22:45:19.205634 24290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.194:33227
I20260926 22:45:19.208716 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24481
I20260926 22:45:19.208792 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:45:19.208927 15075 external_mini_cluster.cc:1428] Setting key 47cb4c7156f8ecc31ca965616d462644
I20260926 22:45:19.209721 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37161
--builtin_ntp_servers=127.14.184.212:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 22:45:19.288933 24612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.289122 24612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.289142 24612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.290192 24612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.290227 24612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.290257 24612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:45:19.291584 24612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:37161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.291755 24612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.291977 24612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.296831 24617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.296779 24620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.296902 24612 server_base.cc:1034] running on GCE node
W20260926 22:45:19.296778 24618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.297266 24612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.297504 24612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.298704 24612 hybrid_clock.cc:648] HybridClock initialized: now 1790462719298654 us; error 29 us; skew 500 ppm
I20260926 22:45:19.299955 24612 webserver.cc:460] Webserver started at http://127.14.184.195:45163/ using document root <none> and password file <none>
I20260926 22:45:19.300199 24612 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.300246 24612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.300393 24612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.301426 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "bb236f3ac9a14738bd1977d0ad87d09c"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "5efe683f227abd7bf77e658d53d39039"
server_key_iv: "334a5eed6d736dde57ec5ef2821f202b"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.301771 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "bb236f3ac9a14738bd1977d0ad87d09c"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "5efe683f227abd7bf77e658d53d39039"
server_key_iv: "334a5eed6d736dde57ec5ef2821f202b"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.303071 24612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:45:19.303784 24626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.304013 24612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:45:19.304127 24612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "bb236f3ac9a14738bd1977d0ad87d09c"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "5efe683f227abd7bf77e658d53d39039"
server_key_iv: "334a5eed6d736dde57ec5ef2821f202b"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.304203 24612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.319617 24612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.319892 24612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.320039 24612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.320247 24612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:19.320567 24612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:19.320631 24612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.320663 24612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:19.320690 24612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.326440 24612 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:44113
I20260926 22:45:19.326486 24739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:44113 every 8 connection(s)
I20260926 22:45:19.326862 24612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:45:19.331640 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.327542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:34237 (local address 127.14.184.254:37161)
0926 22:45:19.327603 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.327605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.327825 (+ 220us) server_negotiation.cc:407] Connection header received
0926 22:45:19.328275 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.328276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.328285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.328355 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.328957 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.329495 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.330551 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.330747 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.330799 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:19.331450 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.331451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.331451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.331455 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.331469 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.331473 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.331513 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.331564 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.331585 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:45:19.331847 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.327512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:37161 (local address 127.14.184.195:34237)
0926 22:45:19.327696 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:19.327742 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:19.328197 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:19.328408 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:19.328411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:19.328418 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:19.328890 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:19.328893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.329573 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.329574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:19.330480 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:19.330482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:19.330805 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:19.330806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:19.330910 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:19.331328 (+ 418us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:45:19.331329 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:19.331354 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:45:19.331374 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:19.331516 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:19.331518 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:19.331520 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:45:19.331564 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 22:45:19.331638 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260926 22:45:19.332427 24740 heartbeater.cc:347] Connected to a master server at 127.14.184.254:37161
I20260926 22:45:19.332527 24740 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:19.332710 24740 heartbeater.cc:511] Master 127.14.184.254:37161 requested a full tablet report, sending...
I20260926 22:45:19.333052 24290 ts_manager.cc:194] Registered new tserver with Master: bb236f3ac9a14738bd1977d0ad87d09c (127.14.184.195:44113)
I20260926 22:45:19.333381 24290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.195:34237
I20260926 22:45:19.334275 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24612
I20260926 22:45:19.334352 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:45:19.334491 15075 external_mini_cluster.cc:1428] Setting key 74d4421508509751dd544fa779f9ba13
I20260926 22:45:19.335402 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 22:45:19.337095 24347 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:19.336696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.14.184.254:37161)
0926 22:45:19.336738 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.336740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.336855 (+ 115us) server_negotiation.cc:407] Connection header received
0926 22:45:19.336901 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.336901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.336910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.336954 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.337051 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43016: BlockingRecv error: recv got EOF from 127.0.0.1:43016 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:45:19.337167 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24350
I20260926 22:45:19.342000 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24481
I20260926 22:45:19.346154 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24612
I20260926 22:45:19.350435 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24259
2026-09-26T22:45:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (559 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T22:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:45:19Z Disabled control of system clock
I20260926 22:45:19.376338 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45013
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:45013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 22:45:19.455116 24751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.455299 24751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.455319 24751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.456423 24751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:45:19.456463 24751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.456476 24751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.456488 24751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:45:19.456501 24751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:45:19.457823 24751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:45013
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.458062 24751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.458359 24751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.462893 24756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.462909 24759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.462898 24757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.463124 24751 server_base.cc:1034] running on GCE node
I20260926 22:45:19.463377 24751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.463681 24751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.464823 24751 hybrid_clock.cc:648] HybridClock initialized: now 1790462719464815 us; error 33 us; skew 500 ppm
I20260926 22:45:19.465962 24751 webserver.cc:460] Webserver started at http://127.14.184.254:43365/ using document root <none> and password file <none>
I20260926 22:45:19.466207 24751 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.466274 24751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.466435 24751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.467489 24751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "ffd8502fa95646dfac21198f64a164b5"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "a4c40d54ed4367a988bf2e33a6137802"
server_key_iv: "1cdbf73d693732dc56ec679d3ea5ee5e"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.467842 24751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "ffd8502fa95646dfac21198f64a164b5"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "a4c40d54ed4367a988bf2e33a6137802"
server_key_iv: "1cdbf73d693732dc56ec679d3ea5ee5e"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.469107 24751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:45:19.469813 24765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.470036 24751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:19.470134 24751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "ffd8502fa95646dfac21198f64a164b5"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "a4c40d54ed4367a988bf2e33a6137802"
server_key_iv: "1cdbf73d693732dc56ec679d3ea5ee5e"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.470218 24751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.473424 24751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.473664 24751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.473809 24751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.477524 24751 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:45013
I20260926 22:45:19.477555 24817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:45013 every 8 connection(s)
I20260926 22:45:19.477902 24751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:45:19.478499 24818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:19.480242 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24751
I20260926 22:45:19.480321 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:45:19.480516 15075 external_mini_cluster.cc:1428] Setting key 8eee277ec7694d83a29504198c395228
I20260926 22:45:19.481570 24818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5: Bootstrap starting.
I20260926 22:45:19.482249 24818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:19.482584 24818 log.cc:824] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:19.483140 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.481030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.14.184.254:45013)
0926 22:45:19.481233 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.481241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.481274 (+ 33us) server_negotiation.cc:407] Connection header received
0926 22:45:19.481792 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.481800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.481851 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.482012 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.482639 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.482642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.482643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.482646 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.482748 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.482759 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.482846 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.482910 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.482963 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260926 22:45:19.483376 24818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5: No bootstrap required, opened a new log
I20260926 22:45:19.484776 24818 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd8502fa95646dfac21198f64a164b5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45013 } }
I20260926 22:45:19.484896 24818 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:19.484916 24818 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd8502fa95646dfac21198f64a164b5, State: Initialized, Role: FOLLOWER
I20260926 22:45:19.485054 24818 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd8502fa95646dfac21198f64a164b5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45013 } }
I20260926 22:45:19.485123 24818 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:45:19.485164 24818 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:45:19.485209 24818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:19.485939 24818 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd8502fa95646dfac21198f64a164b5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45013 } }
I20260926 22:45:19.486047 24818 leader_election.cc:302] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffd8502fa95646dfac21198f64a164b5; no voters:
I20260926 22:45:19.486191 24818 leader_election.cc:288] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:45:19.486253 24823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:45:19.486402 24823 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [term 1 LEADER]: Becoming Leader. State: Replica: ffd8502fa95646dfac21198f64a164b5, State: Running, Role: LEADER
I20260926 22:45:19.486441 24818 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:45:19.486496 24823 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd8502fa95646dfac21198f64a164b5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45013 } }
I20260926 22:45:19.486865 24824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffd8502fa95646dfac21198f64a164b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd8502fa95646dfac21198f64a164b5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45013 } } }
I20260926 22:45:19.486867 24825 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd8502fa95646dfac21198f64a164b5. Latest consensus state: current_term: 1 leader_uuid: "ffd8502fa95646dfac21198f64a164b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd8502fa95646dfac21198f64a164b5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45013 } } }
I20260926 22:45:19.487001 24824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:19.487005 24825 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5 [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:19.487249 24832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:45:19.487730 24832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:45:19.489209 24832 catalog_manager.cc:1347] Generated new cluster ID: b45ea8e3aa534b94960a6156917da6f3
I20260926 22:45:19.489270 24832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:45:19.507870 24832 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:45:19.508024 24832 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:45:19.513608 24832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffd8502fa95646dfac21198f64a164b5: Generated new TSK 0
I20260926 22:45:19.513767 24832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:45:19.515097 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45013
--builtin_ntp_servers=127.14.184.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 22:45:19.606190 24842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.606438 24842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.606477 24842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.607910 24842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.607972 24842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.608018 24842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:45:19.609375 24842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.609633 24842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.609921 24842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.615783 24848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.615782 24850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.615903 24847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.616068 24842 server_base.cc:1034] running on GCE node
I20260926 22:45:19.616400 24842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.616680 24842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.617836 24842 hybrid_clock.cc:648] HybridClock initialized: now 1790462719617816 us; error 36 us; skew 500 ppm
I20260926 22:45:19.619261 24842 webserver.cc:460] Webserver started at http://127.14.184.193:40671/ using document root <none> and password file <none>
I20260926 22:45:19.619508 24842 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.619553 24842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.619697 24842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.620752 24842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "e028c757823048978dfabc625ec2ea22"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "6e870673044ea95e88c219b292b30486"
server_key_iv: "09035dc3293294d9073093077b336eee"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.621150 24842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "e028c757823048978dfabc625ec2ea22"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "6e870673044ea95e88c219b292b30486"
server_key_iv: "09035dc3293294d9073093077b336eee"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.622479 24842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:45:19.623344 24856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.623495 24842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:45:19.623579 24842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "e028c757823048978dfabc625ec2ea22"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "6e870673044ea95e88c219b292b30486"
server_key_iv: "09035dc3293294d9073093077b336eee"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.623651 24842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.626926 24842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.627190 24842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.627331 24842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.627563 24842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:19.627856 24842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:19.627888 24842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.627942 24842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:19.627969 24842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.633606 24842 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:41167
I20260926 22:45:19.633648 24969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:41167 every 8 connection(s)
I20260926 22:45:19.633953 24842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:45:19.636654 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.634644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:48699 (local address 127.14.184.254:45013)
0926 22:45:19.634734 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.634736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.634969 (+ 233us) server_negotiation.cc:407] Connection header received
0926 22:45:19.635400 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.635401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.635406 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.635461 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.636485 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.636486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.636486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.636490 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.636503 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.636507 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.636550 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.636594 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.636605 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 22:45:19.636838 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.634620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45013 (local address 127.14.184.193:48699)
0926 22:45:19.634845 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:19.634883 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:19.635320 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:19.635519 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:19.635522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:19.635530 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:19.636358 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:45:19.636359 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:19.636383 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:45:19.636409 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:19.636549 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:19.636551 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:19.636553 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:45:19.636601 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 22:45:19.636644 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":134,"threads_started":1}
I20260926 22:45:19.637375 24970 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45013
I20260926 22:45:19.637452 24970 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:19.637596 24970 heartbeater.cc:511] Master 127.14.184.254:45013 requested a full tablet report, sending...
I20260926 22:45:19.638075 24782 ts_manager.cc:194] Registered new tserver with Master: e028c757823048978dfabc625ec2ea22 (127.14.184.193:41167)
I20260926 22:45:19.641467 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24842
I20260926 22:45:19.641542 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:45:19.641806 15075 external_mini_cluster.cc:1428] Setting key 44ad2c592e648374a2e83398b8992eac
I20260926 22:45:19.642653 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45013
--builtin_ntp_servers=127.14.184.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 22:45:19.722040 24973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.722405 24973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.722452 24973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.723958 24973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.724021 24973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.724068 24973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:45:19.725406 24973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.725677 24973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.726034 24973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.730892 24979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.730926 24981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.730897 24978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.731055 24973 server_base.cc:1034] running on GCE node
I20260926 22:45:19.731366 24973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.731617 24973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.732765 24973 hybrid_clock.cc:648] HybridClock initialized: now 1790462719732738 us; error 36 us; skew 500 ppm
I20260926 22:45:19.734001 24973 webserver.cc:460] Webserver started at http://127.14.184.194:44185/ using document root <none> and password file <none>
I20260926 22:45:19.734251 24973 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.734297 24973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.734459 24973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.735553 24973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "02b6d9f6d9374d338337aa97101e40ef"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "1ce29862b428bcdd0b6d0fa862aa5f9d"
server_key_iv: "731b09db7db9776d9e3377aa39ed488f"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.735973 24973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "02b6d9f6d9374d338337aa97101e40ef"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "1ce29862b428bcdd0b6d0fa862aa5f9d"
server_key_iv: "731b09db7db9776d9e3377aa39ed488f"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.737337 24973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:45:19.738059 24987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.738255 24973 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:19.738350 24973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "02b6d9f6d9374d338337aa97101e40ef"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "1ce29862b428bcdd0b6d0fa862aa5f9d"
server_key_iv: "731b09db7db9776d9e3377aa39ed488f"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.738425 24973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.741720 24973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.741977 24973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.742102 24973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.742311 24973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:19.742599 24973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:19.742630 24973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.742714 24973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:19.742756 24973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.748369 24973 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:34197
I20260926 22:45:19.748435 25100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:34197 every 8 connection(s)
I20260926 22:45:19.748746 24973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:45:19.751521 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.749449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:39197 (local address 127.14.184.254:45013)
0926 22:45:19.749508 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.749510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.749700 (+ 190us) server_negotiation.cc:407] Connection header received
0926 22:45:19.750104 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.750105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.750110 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.750176 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.751317 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.751319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.751319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.751323 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.751338 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.751342 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.751406 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.751456 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.751469 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:45:19.751713 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.749412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45013 (local address 127.14.184.194:39197)
0926 22:45:19.749579 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:19.749616 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:19.750028 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:19.750227 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:19.750230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:19.750237 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:19.751173 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:45:19.751173 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:19.751200 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:45:19.751226 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:19.751401 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:19.751403 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:19.751405 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:45:19.751454 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 22:45:19.751498 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260926 22:45:19.752197 25101 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45013
I20260926 22:45:19.752270 25101 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:19.752411 25101 heartbeater.cc:511] Master 127.14.184.254:45013 requested a full tablet report, sending...
I20260926 22:45:19.752808 24782 ts_manager.cc:194] Registered new tserver with Master: 02b6d9f6d9374d338337aa97101e40ef (127.14.184.194:34197)
I20260926 22:45:19.757252 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 24973
I20260926 22:45:19.757391 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:45:19.757596 15075 external_mini_cluster.cc:1428] Setting key 36c8b2489e0296f721472582488075b7
I20260926 22:45:19.758474 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45013
--builtin_ntp_servers=127.14.184.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 22:45:19.839493 25104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:19.839686 25104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:19.839707 25104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:19.840819 25104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:19.840890 25104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:19.840919 25104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:45:19.842299 25104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:19.842523 25104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:19.842824 25104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:19.847429 25112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.847472 25109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:19.847486 25110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:19.847766 25104 server_base.cc:1034] running on GCE node
I20260926 22:45:19.847986 25104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:19.848214 25104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:19.849377 25104 hybrid_clock.cc:648] HybridClock initialized: now 1790462719849361 us; error 28 us; skew 500 ppm
I20260926 22:45:19.850590 25104 webserver.cc:460] Webserver started at http://127.14.184.195:35821/ using document root <none> and password file <none>
I20260926 22:45:19.850859 25104 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:19.850929 25104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:19.851083 25104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:19.852104 25104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "6b7345ba79fc4652a57d557dfea9d0c1"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "4e8d7690b732d7cc6854a818557fc792"
server_key_iv: "048fcbad6a849e2ea978b76b5afd66e0"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.852458 25104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "6b7345ba79fc4652a57d557dfea9d0c1"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "4e8d7690b732d7cc6854a818557fc792"
server_key_iv: "048fcbad6a849e2ea978b76b5afd66e0"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.853715 25104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:45:19.854372 25118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.854542 25104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:19.854635 25104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "6b7345ba79fc4652a57d557dfea9d0c1"
format_stamp: "Formatted at 2026-09-26 22:45:19 on dist-test-slave-3986"
server_key: "4e8d7690b732d7cc6854a818557fc792"
server_key_iv: "048fcbad6a849e2ea978b76b5afd66e0"
server_key_version: "encryptionkey@0"
I20260926 22:45:19.854789 25104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:19.857940 25104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:19.858186 25104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:19.858316 25104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:19.858518 25104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:19.858868 25104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:19.858920 25104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.858958 25104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:19.858986 25104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:19.864535 25104 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:38911
I20260926 22:45:19.864583 25231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:38911 every 8 connection(s)
I20260926 22:45:19.864912 25104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:45:19.867640 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.865586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:59247 (local address 127.14.184.254:45013)
0926 22:45:19.865629 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.865631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.865862 (+ 231us) server_negotiation.cc:407] Connection header received
0926 22:45:19.866265 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.866265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.866270 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.866331 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.867426 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:19.867427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:19.867427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:45:19.867431 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:19.867444 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:45:19.867448 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:19.867501 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:19.867577 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 22:45:19.867590 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:45:19.867847 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:19.865550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45013 (local address 127.14.184.195:59247)
0926 22:45:19.865733 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:19.865778 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:19.866188 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:19.866382 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:19.866385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:19.866392 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:19.867296 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:45:19.867297 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:19.867322 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:45:19.867347 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:19.867501 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:19.867503 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:19.867505 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:45:19.867572 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 22:45:19.867615 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260926 22:45:19.868319 25232 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45013
I20260926 22:45:19.868412 25232 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:19.868546 25232 heartbeater.cc:511] Master 127.14.184.254:45013 requested a full tablet report, sending...
I20260926 22:45:19.868961 24782 ts_manager.cc:194] Registered new tserver with Master: 6b7345ba79fc4652a57d557dfea9d0c1 (127.14.184.195:38911)
I20260926 22:45:19.872969 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 25104
I20260926 22:45:19.873072 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:45:19.873327 15075 external_mini_cluster.cc:1428] Setting key 64a75cba9d18fde6427e82327f55edb8
I20260926 22:45:19.874030 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 22:45:19.875895 24821 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:45:19.875345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.14.184.254:45013)
0926 22:45:19.875417 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:19.875419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:19.875530 (+ 111us) server_negotiation.cc:407] Connection header received
0926 22:45:19.875599 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:19.875600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:19.875605 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:19.875669 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:19.875832 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52808: BlockingRecv error: recv got EOF from 127.0.0.1:52808 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:45:19.875944 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24842
I20260926 22:45:19.881076 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24973
I20260926 22:45:19.885187 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 25104
I20260926 22:45:19.889213 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 24751
2026-09-26T22:45:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (538 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:45:19 dist-test-slave-3986 krb5kdc[25242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:45:19 dist-test-slave-3986 krb5kdc[25242](info): set up 2 sockets
Sep 26 22:45:19 dist-test-slave-3986 krb5kdc[25242](info): commencing operation
krb5kdc: starting...
W20260926 22:45:20.911922 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:45:20 dist-test-slave-3986 krb5kdc[25242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:45:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:45:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:45:21.025074 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:42515
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:21.101980 25255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:21.102203 25255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:21.102236 25255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:21.103338 25255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:45:21.103402 25255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:21.103425 25255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:21.103444 25255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:45:21.103472 25255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:45:21.104908 25255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:21.105163 25255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:21.105373 25255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:21.110287 25260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:21.110308 25261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:21.110312 25255 server_base.cc:1034] running on GCE node
W20260926 22:45:21.110347 25263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:21.110774 25255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:21.111075 25255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:21.112254 25255 hybrid_clock.cc:648] HybridClock initialized: now 1790462721112247 us; error 36 us; skew 500 ppm
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:21.114845 25255 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:45:21.115275 25255 webserver.cc:460] Webserver started at http://127.14.184.254:45421/ using document root <none> and password file <none>
I20260926 22:45:21.115504 25255 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:21.115571 25255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:21.115706 25255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:21.116724 25255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "2f90c1033552491195f7c6236661a6a5"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "4487c457c524596ded17f269ea17483e"
server_key_iv: "8ebe3291d09156b5c0d98e69d150ae76"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.117089 25255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "2f90c1033552491195f7c6236661a6a5"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "4487c457c524596ded17f269ea17483e"
server_key_iv: "8ebe3291d09156b5c0d98e69d150ae76"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.118382 25255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:45:21.119226 25270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.119408 25255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.119505 25255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "2f90c1033552491195f7c6236661a6a5"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "4487c457c524596ded17f269ea17483e"
server_key_iv: "8ebe3291d09156b5c0d98e69d150ae76"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.119592 25255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:21.135342 25255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:21.137183 25255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:21.137346 25255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:21.141443 25255 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:42515
I20260926 22:45:21.141583 25322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:42515 every 8 connection(s)
I20260926 22:45:21.141878 25255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:45:21.142971 25323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:21.145535 25323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5: Bootstrap starting.
I20260926 22:45:21.146127 25323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:21.146432 25323 log.cc:824] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:21.147231 25323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5: No bootstrap required, opened a new log
I20260926 22:45:21.148844 25323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f90c1033552491195f7c6236661a6a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42515 } }
I20260926 22:45:21.148980 25323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:21.149003 25323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f90c1033552491195f7c6236661a6a5, State: Initialized, Role: FOLLOWER
I20260926 22:45:21.149158 25323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f90c1033552491195f7c6236661a6a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42515 } }
I20260926 22:45:21.149253 25323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:45:21.149291 25323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:45:21.149359 25323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:21.149356 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 25255
I20260926 22:45:21.149436 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:45:21.149616 15075 external_mini_cluster.cc:1428] Setting key 6eadee7def0e7347c73dd843c03d6214
I20260926 22:45:21.150506 25323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f90c1033552491195f7c6236661a6a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42515 } }
I20260926 22:45:21.150616 25323 leader_election.cc:302] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f90c1033552491195f7c6236661a6a5; no voters:
I20260926 22:45:21.150784 25323 leader_election.cc:288] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:45:21.150830 25328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:45:21.150981 25328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f90c1033552491195f7c6236661a6a5, State: Running, Role: LEADER
I20260926 22:45:21.151068 25323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:45:21.151085 25328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f90c1033552491195f7c6236661a6a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42515 } }
I20260926 22:45:21.151420 25328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f90c1033552491195f7c6236661a6a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f90c1033552491195f7c6236661a6a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42515 } } }
I20260926 22:45:21.151510 25328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:21.151644 25328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f90c1033552491195f7c6236661a6a5. Latest consensus state: current_term: 1 leader_uuid: "2f90c1033552491195f7c6236661a6a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f90c1033552491195f7c6236661a6a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42515 } } }
I20260926 22:45:21.151742 25328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5 [sys.catalog]: This master's current role is: LEADER
I20260926 22:45:21.151878 25335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:45:21.152594 25335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:45:21.154295 25335 catalog_manager.cc:1347] Generated new cluster ID: 34e75b842109415e8c35d65c2aa6b2f1
I20260926 22:45:21.154345 25335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:21.161023 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.150116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.14.184.254:42515)
0926 22:45:21.150319 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.150328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.150367 (+ 39us) server_negotiation.cc:407] Connection header received
0926 22:45:21.150484 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.150491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.150558 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.150730 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:21.151566 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.152982 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.153896 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.154473 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.154575 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.157458 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:21.157462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:21.157462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:21.157473 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:21.159417 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.159817 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.159820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.159820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.159930 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.160177 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.160178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.160178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.160450 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:21.160601 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.160717 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.160839 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260926 22:45:21.162812 25335 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:45:21.163471 25335 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:45:21.178507 25335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5: Generated new TSK 0
I20260926 22:45:21.178700 25335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:45:21.209326 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42515
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:21.288704 25349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:21.288928 25349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:21.288964 25349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:21.290112 25349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:21.290179 25349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:21.290221 25349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:45:21.291664 25349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:21.291916 25349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:21.292131 25349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:21.296938 25357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:21.296947 25355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:21.296994 25354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:21.297111 25349 server_base.cc:1034] running on GCE node
I20260926 22:45:21.297379 25349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:21.297621 25349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:21.298781 25349 hybrid_clock.cc:648] HybridClock initialized: now 1790462721298785 us; error 46 us; skew 500 ppm
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:21.301343 25349 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:45:21.301705 25349 webserver.cc:460] Webserver started at http://127.14.184.193:43263/ using document root <none> and password file <none>
I20260926 22:45:21.301889 25349 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:21.301929 25349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:21.302011 25349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:21.302994 25349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "02b6504da750425d50b69b047e3a4643"
server_key_iv: "84294448ad9e83c4282f23b98f0f4bc6"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.303272 25349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "02b6504da750425d50b69b047e3a4643"
server_key_iv: "84294448ad9e83c4282f23b98f0f4bc6"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.304515 25349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:45:21.305145 25364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.305353 25349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:21.305429 25349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "02b6504da750425d50b69b047e3a4643"
server_key_iv: "84294448ad9e83c4282f23b98f0f4bc6"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.305480 25349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:21.320402 25349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:21.322171 25349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:21.322286 25349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:21.322451 25349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:21.322799 25349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:21.322835 25349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.322881 25349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:21.322906 25349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.328514 25349 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:43729
I20260926 22:45:21.328613 25477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:43729 every 8 connection(s)
I20260926 22:45:21.328899 25349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:45:21.334364 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 25349
I20260926 22:45:21.334494 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:45:21.334779 15075 external_mini_cluster.cc:1428] Setting key 289c7a678d7a68777a9cb12e54106c69
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:21.339187 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.329587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:41231 (local address 127.14.184.254:42515)
0926 22:45:21.329657 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.329659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.329867 (+ 208us) server_negotiation.cc:407] Connection header received
0926 22:45:21.330373 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.330375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.330390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.330446 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:21.331430 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.331930 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.333231 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.333547 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.333628 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.336275 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:21.336276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:21.336276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:21.336283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:21.337730 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.338129 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.338130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.338131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.338194 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.338463 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.338464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.338464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.338602 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:21.338672 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.339002 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.339100 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 22:45:21.339342 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.329577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:42515 (local address 127.14.184.193:41231)
0926 22:45:21.329744 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.329776 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.330265 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.330519 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.330522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.331145 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:21.331329 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.331334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.332025 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.332026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.333133 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.333136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.333657 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.333658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.333785 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.334332 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:21.334340 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:21.336174 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:21.337826 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:21.337833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:21.337835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:21.338051 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:21.338273 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:21.338274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:21.338274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:21.338392 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:21.338726 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:21.338732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:21.338858 (+ 126us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.338964 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.339114 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260926 22:45:21.339923 25478 heartbeater.cc:347] Connected to a master server at 127.14.184.254:42515
I20260926 22:45:21.340029 25478 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:21.340225 25478 heartbeater.cc:511] Master 127.14.184.254:42515 requested a full tablet report, sending...
I20260926 22:45:21.340790 25287 ts_manager.cc:194] Registered new tserver with Master: 5407ab7b07dc4feca2ae810ae3c4b6b9 (127.14.184.193:43729)
I20260926 22:45:21.341712 25287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:41231
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:45:21.359311 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42515
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:21.439167 25483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:21.439360 25483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:21.439383 25483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:21.440478 25483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:21.440526 25483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:21.440557 25483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:45:21.441915 25483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:21.442087 25483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:21.442246 25483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:21.447284 25489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:21.447289 25491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:21.447314 25488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:21.447333 25483 server_base.cc:1034] running on GCE node
I20260926 22:45:21.447616 25483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:21.447865 25483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:21.449023 25483 hybrid_clock.cc:648] HybridClock initialized: now 1790462721449010 us; error 31 us; skew 500 ppm
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:21.451582 25483 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:45:21.452124 25483 webserver.cc:460] Webserver started at http://127.14.184.194:34329/ using document root <none> and password file <none>
I20260926 22:45:21.452351 25483 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:21.452417 25483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:21.452565 25483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:21.453562 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "12b001c82702411ca0a7ed799343d91a"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "b000610887a080116200c83e8303c8ab"
server_key_iv: "36d1d2ce11f15baaf4be23ae1c13d10f"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.453922 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "12b001c82702411ca0a7ed799343d91a"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "b000610887a080116200c83e8303c8ab"
server_key_iv: "36d1d2ce11f15baaf4be23ae1c13d10f"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.455291 25483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:45:21.455994 25498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.456166 25483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:21.456259 25483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "12b001c82702411ca0a7ed799343d91a"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "b000610887a080116200c83e8303c8ab"
server_key_iv: "36d1d2ce11f15baaf4be23ae1c13d10f"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.456338 25483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:21.517271 25483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:21.519749 25483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:21.519922 25483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:21.520162 25483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:21.520474 25483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:21.520527 25483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.520562 25483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:21.520588 25483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.526338 25483 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:42967
I20260926 22:45:21.526425 25611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:42967 every 8 connection(s)
I20260926 22:45:21.526793 25483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:21.534576 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 25483
I20260926 22:45:21.534734 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:45:21.534961 15075 external_mini_cluster.cc:1428] Setting key 9a2a4b22ad8aaa3b482ae214a929e281
I20260926 22:45:21.535691 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.527521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:47141 (local address 127.14.184.254:42515)
0926 22:45:21.527578 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.527581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.527813 (+ 232us) server_negotiation.cc:407] Connection header received
0926 22:45:21.528259 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.528261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.528271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.528325 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:21.529232 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.529716 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.530699 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.530898 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.530963 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.533050 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:21.533051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:21.533051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:21.533058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:21.534207 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.534618 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.534619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.534620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.534724 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.535039 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.535040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.535040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.535168 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:21.535240 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.535521 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.535606 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:45:21.535864 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.527506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:42515 (local address 127.14.184.194:47141)
0926 22:45:21.527661 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.527697 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.528163 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.528394 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.528398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.528947 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:21.529139 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.529143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.529804 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.529804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.530589 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.530591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.530976 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.530977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.531077 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.531596 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:21.531605 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:21.532949 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:21.534304 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:21.534310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:21.534312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:21.534534 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:21.534840 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:21.534840 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:21.534841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:21.534948 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:21.535241 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:21.535248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:21.535385 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.535486 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.535632 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260926 22:45:21.536445 25612 heartbeater.cc:347] Connected to a master server at 127.14.184.254:42515
I20260926 22:45:21.536526 25612 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:21.536711 25612 heartbeater.cc:511] Master 127.14.184.254:42515 requested a full tablet report, sending...
I20260926 22:45:21.537096 25287 ts_manager.cc:194] Registered new tserver with Master: 12b001c82702411ca0a7ed799343d91a (127.14.184.194:42967)
I20260926 22:45:21.537488 25287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:47141
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:45:21.558548 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42515
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:45:21.636974 25617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:45:21.637205 25617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:45:21.637226 25617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:45:21.638361 25617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:45:21.638408 25617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:45:21.638434 25617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:45:21.639794 25617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:45:21.639959 25617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:45:21.640125 25617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:45:21.645053 25622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:21.645174 25617 server_base.cc:1034] running on GCE node
W20260926 22:45:21.645078 25625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:45:21.645063 25623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:45:21.645428 25617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:45:21.645625 25617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:45:21.646754 25617 hybrid_clock.cc:648] HybridClock initialized: now 1790462721646743 us; error 28 us; skew 500 ppm
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:45:21.649147 25617 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:45:21.649575 25617 webserver.cc:460] Webserver started at http://127.14.184.195:33805/ using document root <none> and password file <none>
I20260926 22:45:21.649782 25617 fs_manager.cc:359] Metadata directory not provided
I20260926 22:45:21.649856 25617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:45:21.650000 25617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:45:21.651100 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "0e41da8845094c3b977661e3b8702548"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "412316cc37c5a7bce49499705f428036"
server_key_iv: "fbb4955c496cd54607c3b6ed515d99c0"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.651547 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "0e41da8845094c3b977661e3b8702548"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "412316cc37c5a7bce49499705f428036"
server_key_iv: "fbb4955c496cd54607c3b6ed515d99c0"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.652997 25617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:45:21.653743 25632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.653931 25617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:45:21.654013 25617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "0e41da8845094c3b977661e3b8702548"
format_stamp: "Formatted at 2026-09-26 22:45:21 on dist-test-slave-3986"
server_key: "412316cc37c5a7bce49499705f428036"
server_key_iv: "fbb4955c496cd54607c3b6ed515d99c0"
server_key_version: "encryptionkey@0"
I20260926 22:45:21.654084 25617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:45:21.663775 25617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:45:21.665474 25617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:45:21.665580 25617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:45:21.665747 25617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:45:21.665984 25617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:45:21.666013 25617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.666034 25617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:45:21.666054 25617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:45:21.671609 25617 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:44503
I20260926 22:45:21.671721 25745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:44503 every 8 connection(s)
I20260926 22:45:21.671936 25617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:45:21.672909 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 25617
I20260926 22:45:21.673002 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:45:21.673179 15075 external_mini_cluster.cc:1428] Setting key 6b093ce61def8d96cebeb35a7568aa1c
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:45:21.681106 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.672676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41609 (local address 127.14.184.254:42515)
0926 22:45:21.672756 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.672758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.672940 (+ 182us) server_negotiation.cc:407] Connection header received
0926 22:45:21.673549 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.673550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.673561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.673629 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:21.674725 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.675218 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.676160 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.676382 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.676446 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.678443 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:21.678444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:21.678445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:21.678451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:21.679992 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.680356 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.680356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.680357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.680406 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.680648 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.680648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.680649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.680740 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:21.680788 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.680990 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.681046 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 22:45:21.681317 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.672645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:42515 (local address 127.14.184.195:41609)
0926 22:45:21.672835 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.672869 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.673468 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.673702 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.673706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.674403 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:45:21.674586 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.674591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.675295 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.675295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.676068 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.676070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.676460 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.676461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.676553 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.677053 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:45:21.677061 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:45:21.678366 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:45:21.680066 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:21.680072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:21.680073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:21.680292 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:21.680485 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:45:21.680486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:45:21.680486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:45:21.680590 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:45:21.680795 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:45:21.680799 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:45:21.680901 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.680978 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.681089 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":123,"threads_started":1}
I20260926 22:45:21.681807 25746 heartbeater.cc:347] Connected to a master server at 127.14.184.254:42515
I20260926 22:45:21.681883 25746 heartbeater.cc:464] Registering TS with master...
I20260926 22:45:21.682065 25746 heartbeater.cc:511] Master 127.14.184.254:42515 requested a full tablet report, sending...
I20260926 22:45:21.682428 25287 ts_manager.cc:194] Registered new tserver with Master: 0e41da8845094c3b977661e3b8702548 (127.14.184.195:44503)
I20260926 22:45:21.682874 25287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:41609
I20260926 22:45:21.684985 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:45:21.691817 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.686228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.14.184.254:42515)
0926 22:45:21.686274 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.686276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.686406 (+ 130us) server_negotiation.cc:407] Connection header received
0926 22:45:21.686451 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.686452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.686460 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.686504 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:45:21.687076 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.687503 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.688257 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.688432 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.688480 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.689546 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:45:21.689547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:45:21.689547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:45:21.689552 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:45:21.690860 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.691169 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.691170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.691170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.691217 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:45:21.691384 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:45:21.691385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:45:21.691385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:45:21.691507 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:45:21.691594 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.691707 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.691760 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 22:45:21.693272 25287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:45:21.694855 25287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:45:21.701108 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.697055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43729 (local address 127.0.0.1:47324)
0926 22:45:21.697227 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.697263 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.697323 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.697627 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.697630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.697632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:45:21.697704 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.697706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.699028 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.699029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.700099 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.700100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.700778 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.700878 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.700892 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.700962 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.701031 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":5,"thread_start_us":71,"threads_started":1}
I20260926 22:45:21.701272 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.697260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:44503 (local address 127.0.0.1:48248)
0926 22:45:21.697460 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.697469 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.697512 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.697868 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.697869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.697871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:45:21.697914 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.697916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.698778 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.698779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.700025 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.700027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.700982 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.701060 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.701081 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.701132 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.701175 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260926 22:45:21.701350 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.697261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.14.184.193:43729)
0926 22:45:21.697449 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.697453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.697463 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:45:21.697509 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.697511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.697528 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.697621 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:45:21.697770 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.698926 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700207 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.700686 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700779 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.700842 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.701213 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.701267 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260926 22:45:21.701488 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.697453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.14.184.195:44503)
0926 22:45:21.697664 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.697668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.697676 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:21.697709 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.697710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.697725 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.697791 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:45:21.697968 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.698621 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700172 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.700835 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700894 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.700945 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.701365 (+ 420us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.701410 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260926 22:45:21.701525 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.697055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:42967 (local address 127.0.0.1:35094)
0926 22:45:21.697419 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.697428 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.697514 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.697797 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.697798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.697800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:45:21.697843 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.697845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.699112 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.699113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.699928 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.699931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.700982 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.701060 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.701089 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.701132 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.701175 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":30,"thread_start_us":66,"threads_started":1}
I20260926 22:45:21.701803 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.697178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.14.184.194:42967)
0926 22:45:21.697377 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.697382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.697488 (+ 106us) server_negotiation.cc:407] Connection header received
0926 22:45:21.697617 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.697620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.697638 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.697711 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:45:21.697927 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.699006 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700023 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.700790 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.700868 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.700945 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.701680 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.701729 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260926 22:45:21.702013 25412 tablet_service.cc:1467] Processing CreateTablet for tablet d7a24d233438498aa2bfb515876f4984 (DEFAULT_TABLE table=test-table [id=f9a76c5d8daf47b7bb1e73ec3bceb469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:45:21.702356 25412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a24d233438498aa2bfb515876f4984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:21.702394 25546 tablet_service.cc:1467] Processing CreateTablet for tablet d7a24d233438498aa2bfb515876f4984 (DEFAULT_TABLE table=test-table [id=f9a76c5d8daf47b7bb1e73ec3bceb469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:45:21.702735 25546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a24d233438498aa2bfb515876f4984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:21.703032 25680 tablet_service.cc:1467] Processing CreateTablet for tablet d7a24d233438498aa2bfb515876f4984 (DEFAULT_TABLE table=test-table [id=f9a76c5d8daf47b7bb1e73ec3bceb469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:45:21.703353 25680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a24d233438498aa2bfb515876f4984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:45:21.705581 25763 tablet_bootstrap.cc:492] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Bootstrap starting.
I20260926 22:45:21.705895 25765 tablet_bootstrap.cc:492] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Bootstrap starting.
I20260926 22:45:21.706368 25763 tablet_bootstrap.cc:654] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:21.706491 25765 tablet_bootstrap.cc:654] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:21.706828 25764 tablet_bootstrap.cc:492] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: Bootstrap starting.
I20260926 22:45:21.707024 25763 log.cc:824] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:21.707229 25765 log.cc:824] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:21.707526 25764 tablet_bootstrap.cc:654] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: Neither blocks nor log segments found. Creating new log.
I20260926 22:45:21.707841 25763 tablet_bootstrap.cc:492] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: No bootstrap required, opened a new log
I20260926 22:45:21.707926 25763 ts_tablet_manager.cc:1397] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:45:21.708108 25765 tablet_bootstrap.cc:492] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: No bootstrap required, opened a new log
I20260926 22:45:21.708110 25764 log.cc:824] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: Log is configured to *not* fsync() on all Append() calls
I20260926 22:45:21.708175 25765 ts_tablet_manager.cc:1397] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:45:21.709139 25764 tablet_bootstrap.cc:492] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: No bootstrap required, opened a new log
I20260926 22:45:21.709209 25764 ts_tablet_manager.cc:1397] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:45:21.709753 25763 raft_consensus.cc:348] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.709903 25763 raft_consensus.cc:374] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:21.709944 25763 raft_consensus.cc:729] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e41da8845094c3b977661e3b8702548, State: Initialized, Role: FOLLOWER
I20260926 22:45:21.710050 25763 consensus_queue.cc:260] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.710163 25765 raft_consensus.cc:348] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.710314 25765 raft_consensus.cc:374] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:21.710345 25763 ts_tablet_manager.cc:1428] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:45:21.710358 25765 raft_consensus.cc:729] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b001c82702411ca0a7ed799343d91a, State: Initialized, Role: FOLLOWER
I20260926 22:45:21.710474 25746 heartbeater.cc:503] Master 127.14.184.254:42515 was elected leader, sending a full tablet report...
I20260926 22:45:21.710461 25765 consensus_queue.cc:260] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.710749 25765 ts_tablet_manager.cc:1428] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:45:21.710768 25612 heartbeater.cc:503] Master 127.14.184.254:42515 was elected leader, sending a full tablet report...
I20260926 22:45:21.711694 25764 raft_consensus.cc:348] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.711862 25764 raft_consensus.cc:374] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:45:21.711908 25764 raft_consensus.cc:729] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5407ab7b07dc4feca2ae810ae3c4b6b9, State: Initialized, Role: FOLLOWER
I20260926 22:45:21.712126 25764 consensus_queue.cc:260] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.712386 25764 ts_tablet_manager.cc:1428] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:45:21.712414 25478 heartbeater.cc:503] Master 127.14.184.254:42515 was elected leader, sending a full tablet report...
W20260926 22:45:21.777407 25613 tablet.cc:2369] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:45:21.819585 25769 raft_consensus.cc:482] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:45:21.819715 25769 raft_consensus.cc:504] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.820056 25769 leader_election.cc:288] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5407ab7b07dc4feca2ae810ae3c4b6b9 (127.14.184.193:43729), 12b001c82702411ca0a7ed799343d91a (127.14.184.194:42967)
I20260926 22:45:21.823288 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.820195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43729 (local address 127.14.184.195:38809)
0926 22:45:21.820248 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.820261 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.820302 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.820449 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.820450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.820452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:45:21.820530 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.820532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.821503 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.821504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.822380 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.822383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.822996 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.822998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.823089 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.823093 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.823148 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.823193 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":30}
I20260926 22:45:21.823307 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.820253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:38809 (local address 127.14.184.193:43729)
0926 22:45:21.820301 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.820303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.820314 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:45:21.820353 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.820354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.820363 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.820426 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:45:21.820603 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.821390 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.822474 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.822912 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.822975 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.822982 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.823182 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.823224 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:45:21.823397 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.820262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:42967 (local address 127.14.184.195:44341)
0926 22:45:21.820437 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.820447 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:45:21.820502 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:45:21.820650 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:45:21.820651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:45:21.820656 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:45:21.820703 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.820704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.821550 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.821551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.822361 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:45:21.822363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.823112 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.823113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:45:21.823204 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.823208 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:45:21.823282 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 22:45:21.823317 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":119,"threads_started":1}
I20260926 22:45:21.823428 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.820284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:44341 (local address 127.14.184.194:42967)
0926 22:45:21.820341 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.820343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.820512 (+ 169us) server_negotiation.cc:407] Connection header received
0926 22:45:21.820557 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.820558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.820583 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.820640 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:45:21.820767 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.821454 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.822445 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.823025 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.823094 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.823103 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.823295 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.823349 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:45:21.823581 25432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a24d233438498aa2bfb515876f4984" candidate_uuid: "0e41da8845094c3b977661e3b8702548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" is_pre_election: true
I20260926 22:45:21.823699 25566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a24d233438498aa2bfb515876f4984" candidate_uuid: "0e41da8845094c3b977661e3b8702548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12b001c82702411ca0a7ed799343d91a" is_pre_election: true
I20260926 22:45:21.823724 25432 raft_consensus.cc:2454] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e41da8845094c3b977661e3b8702548 in term 0.
I20260926 22:45:21.823819 25566 raft_consensus.cc:2454] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e41da8845094c3b977661e3b8702548 in term 0.
I20260926 22:45:21.823987 25635 leader_election.cc:302] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e41da8845094c3b977661e3b8702548, 5407ab7b07dc4feca2ae810ae3c4b6b9; no voters:
I20260926 22:45:21.824149 25769 raft_consensus.cc:2781] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:45:21.824215 25769 raft_consensus.cc:482] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:45:21.824240 25769 raft_consensus.cc:3037] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:21.824952 25769 raft_consensus.cc:504] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.825095 25769 leader_election.cc:288] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [CANDIDATE]: Term 1 election: Requested vote from peers 5407ab7b07dc4feca2ae810ae3c4b6b9 (127.14.184.193:43729), 12b001c82702411ca0a7ed799343d91a (127.14.184.194:42967)
I20260926 22:45:21.825268 25566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a24d233438498aa2bfb515876f4984" candidate_uuid: "0e41da8845094c3b977661e3b8702548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12b001c82702411ca0a7ed799343d91a"
I20260926 22:45:21.825289 25432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a24d233438498aa2bfb515876f4984" candidate_uuid: "0e41da8845094c3b977661e3b8702548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9"
I20260926 22:45:21.825347 25566 raft_consensus.cc:3037] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:21.825358 25432 raft_consensus.cc:3037] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:45:21.826225 25432 raft_consensus.cc:2454] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e41da8845094c3b977661e3b8702548 in term 1.
I20260926 22:45:21.826308 25566 raft_consensus.cc:2454] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e41da8845094c3b977661e3b8702548 in term 1.
I20260926 22:45:21.826403 25635 leader_election.cc:302] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e41da8845094c3b977661e3b8702548, 5407ab7b07dc4feca2ae810ae3c4b6b9; no voters:
I20260926 22:45:21.826545 25769 raft_consensus.cc:2781] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:45:21.826752 25769 raft_consensus.cc:686] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 1 LEADER]: Becoming Leader. State: Replica: 0e41da8845094c3b977661e3b8702548, State: Running, Role: LEADER
I20260926 22:45:21.826866 25769 consensus_queue.cc:237] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } }
I20260926 22:45:21.827715 25287 catalog_manager.cc:5495] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e41da8845094c3b977661e3b8702548 (127.14.184.195). New cstate: current_term: 1 leader_uuid: "0e41da8845094c3b977661e3b8702548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e41da8845094c3b977661e3b8702548" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44503 } health_report { overall_health: HEALTHY } } }
W20260926 22:45:21.829559 25479 tablet.cc:2369] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:45:21.849594 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.847288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.14.184.195:44503)
0926 22:45:21.847351 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:45:21.847353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:45:21.847361 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:45:21.847423 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:45:21.847423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:45:21.847438 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:45:21.847498 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:45:21.847634 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.848089 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.848807 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:45:21.848990 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:45:21.849045 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:45:21.849180 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:45:21.849278 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:45:21.849338 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:45:21.849444 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 22:45:21.849460 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 22:45:21.851562 25566 raft_consensus.cc:1261] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 1 FOLLOWER]: Refusing update from remote peer 0e41da8845094c3b977661e3b8702548: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:45:21.851562 25432 raft_consensus.cc:1261] T d7a24d233438498aa2bfb515876f4984 P 5407ab7b07dc4feca2ae810ae3c4b6b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e41da8845094c3b977661e3b8702548: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:45:21.851959 25773 consensus_queue.cc:1035] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5407ab7b07dc4feca2ae810ae3c4b6b9" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:45:21.852133 25769 consensus_queue.cc:1035] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12b001c82702411ca0a7ed799343d91a" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:45:21.854252 25776 mvcc.cc:205] Tried to move back new op lower bound from 7333735308700983296 to 7333735308603588608. Current Snapshot: MvccSnapshot[applied={T|T < 7333735308700983296}]
I20260926 22:45:21.854732 25778 mvcc.cc:205] Tried to move back new op lower bound from 7333735308700983296 to 7333735308603588608. Current Snapshot: MvccSnapshot[applied={T|T < 7333735308700983296}]
I20260926 22:45:21.855662 25777 mvcc.cc:205] Tried to move back new op lower bound from 7333735308700983296 to 7333735308603588608. Current Snapshot: MvccSnapshot[applied={T|T < 7333735308700983296 or (T in {7333735308700983296})}]
I20260926 22:45:21.856792 25286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:45:21.856891 25286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:45:21.857614 25286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f90c1033552491195f7c6236661a6a5: Sending DeleteTablet for 3 replicas of tablet d7a24d233438498aa2bfb515876f4984
I20260926 22:45:21.857946 25546 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a24d233438498aa2bfb515876f4984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:21 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:35094
I20260926 22:45:21.857985 25412 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a24d233438498aa2bfb515876f4984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:21 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:47324
I20260926 22:45:21.858148 25680 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a24d233438498aa2bfb515876f4984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:45:21 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:48248
I20260926 22:45:21.858357 25785 tablet_replica.cc:330] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: stopping tablet replica
I20260926 22:45:21.858443 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 25349
I20260926 22:45:21.858479 25785 raft_consensus.cc:2229] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:45:21.862752 25785 raft_consensus.cc:2258] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:45:21.863210 25785 ts_tablet_manager.cc:1905] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:21.865154 25785 ts_tablet_manager.cc:1918] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:45:21.865234 25785 log.cc:1196] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/d7a24d233438498aa2bfb515876f4984
I20260926 22:45:21.865514 25785 ts_tablet_manager.cc:1939] T d7a24d233438498aa2bfb515876f4984 P 12b001c82702411ca0a7ed799343d91a: Deleting consensus metadata
I20260926 22:45:21.866041 25274 catalog_manager.cc:4841] TS 12b001c82702411ca0a7ed799343d91a (127.14.184.194:42967): tablet d7a24d233438498aa2bfb515876f4984 (table test-table [id=f9a76c5d8daf47b7bb1e73ec3bceb469]) successfully deleted
W20260926 22:45:21.866745 25273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:43729 (error 108)
I20260926 22:45:21.867009 25787 tablet_replica.cc:330] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: stopping tablet replica
I20260926 22:45:21.867164 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 25483
I20260926 22:45:21.867245 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:45:21.867127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43729 (local address 127.0.0.1:47336)
0926 22:45:21.867166 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 22:45:21.867214 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:45:21.867913 25787 raft_consensus.cc:2229] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 1 LEADER]: Raft consensus shutting down.
W20260926 22:45:21.867944 25273 catalog_manager.cc:4584] TS 5407ab7b07dc4feca2ae810ae3c4b6b9 (127.14.184.193:43729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7a24d233438498aa2bfb515876f4984: Network error: Client connection negotiation failed: client connection to 127.14.184.193:43729: connect: Connection refused (error 111)
I20260926 22:45:21.868022 25787 raft_consensus.cc:2258] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:45:21.868407 25787 ts_tablet_manager.cc:1905] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:45:21.869663 25787 ts_tablet_manager.cc:1918] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:45:21.869776 25787 log.cc:1196] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/d7a24d233438498aa2bfb515876f4984
I20260926 22:45:21.870074 25787 ts_tablet_manager.cc:1939] T d7a24d233438498aa2bfb515876f4984 P 0e41da8845094c3b977661e3b8702548: Deleting consensus metadata
I20260926 22:45:21.870451 25273 catalog_manager.cc:4841] TS 0e41da8845094c3b977661e3b8702548 (127.14.184.195:44503): tablet d7a24d233438498aa2bfb515876f4984 (table test-table [id=f9a76c5d8daf47b7bb1e73ec3bceb469]) successfully deleted
I20260926 22:45:21.871603 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 25617
I20260926 22:45:21.877038 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 25255
2026-09-26T22:45:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1988 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25791](info): set up 2 sockets
Sep 26 22:45:21 dist-test-slave-3986 krb5kdc[25791](info): commencing operation
krb5kdc: starting...
W20260926 22:45:22.900467 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:45:22 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:45:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/rangeradmin_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/rangeradmin_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/rangerlookup_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/rangerlookup_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
I20260926 22:45:23.074537 15075 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmeYARH/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/postgres -l logfile start
2026-09-26 22:45:25.527 UTC [25821] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 22:45:25.527 UTC [25821] LOG: listening on IPv4 address "127.14.184.212", port 34559
2026-09-26 22:45:25.531 UTC [25821] LOG: listening on Unix socket "/tmp/.s.PGSQL.34559"
2026-09-26 22:45:25.551 UTC [25824] LOG: database system was shut down at 2026-09-26 22:45:24 UTC
2026-09-26 22:45:25.555 UTC [25821] LOG: database system is ready to accept connections
I20260926 22:45:26.507831 15075 mini_postgres.cc:96] Postgres bound to 34559
2026-09-26 22:45:26.513 UTC [25832] FATAL: database "slave" does not exist
127.14.184.212:34559 - accepting connections
I20260926 22:45:26.514145 15075 mini_ranger.cc:161] Starting Ranger...
I20260926 22:45:26.528295 15075 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 22:45:26.998157 15075 mini_ranger.cc:87] Created ranger Postgres database
I20260926 22:45:26.998257 15075 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-admin
2026-09-26 22:45:27,273 [I] DB FLAVOR :POSTGRES
2026-09-26 22:45:27,273 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 22:45:27,273 [I] Checking connection..
2026-09-26 22:45:27,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 22:45:27,520 [I] Checking connection passed.
2026-09-26 22:45:27,521 [I] --------- Verifying version history table ---------
2026-09-26 22:45:27,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 22:45:27,751 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 22:45:27,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 22:45:28,010 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 22:45:28,010 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 22:45:28,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 22:45:28.254 UTC [25896] WARNING: there is no transaction in progress
2026-09-26 22:45:28,261 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 22:45:28,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 22:45:28,516 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 22:45:28,516 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 22:45:28,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 22:45:28,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 22:45:29,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3986.c.gcp-upstream.internal','N') ;"
2026-09-26 22:45:29,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 22:45:29,499 [I] Table x_portal_user does not exist in database ranger
2026-09-26 22:45:29,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 22:45:29,731 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 22:45:29,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 22:45:29,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 22:45:30,241 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 22:45:30,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 22:45:31.183 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.198 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.214 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.229 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.245 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.263 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.314 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.320 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.333 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.343 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.353 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.366 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.375 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.381 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.388 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:31.394 UTC [26004] WARNING: there is no transaction in progress
2026-09-26 22:45:32,019 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 22:45:32,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 22:45:32,264 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 22:45:32,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 22:45:32,518 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 22:45:32,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 22:45:32,769 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 22:45:32,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 22:45:33,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 22:45:33,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3986.c.gcp-upstream.internal';"
2026-09-26 22:45:33,534 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 22:45:33,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 22:45:33,782 [I] DB_PATCHES have already been applied
I20260926 22:45:33.788444 15075 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmeYARH/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 22:45:33.788518 15075 mini_ranger.cc:193] Using host: 127.14.184.212
I20260926 22:45:33.791630 15075 mini_ranger.cc:227] Ranger admin URL: http://127.14.184.212:47881
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 10:45:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 22:45:34.888824 15075 mini_ranger.cc:160] Time spent starting Ranger: real 8.375s user 0.000s sys 0.009s
Sep 26, 2026 10:45:53 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 10:45:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 10:45:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 10:45:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 10:45:54 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 22:45:55.472962 15075 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/rangerkms_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/rangerkms_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.184.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.184.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 22:45:55 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462755, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 22:45:55.564922 15075 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 22:45:55.574379 15075 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 22:45:56.424432 15075 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 22:45:56.424515 15075 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms
I20260926 22:45:56.424535 15075 mini_ranger_kms.cc:227] Using postgres at 127.14.184.212:34559
2026-09-26 22:45:56,652 [I] DB FLAVOR :POSTGRES
2026-09-26 22:45:56,652 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 22:45:56,652 [I] Checking connection
2026-09-26 22:45:56,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 22:45:56,876 [I] connection success
2026-09-26 22:45:56,877 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 22:45:56,877 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 22:45:56,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 22:45:57,114 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 22:45:57,114 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 22:45:57,114 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 22:45:57,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:34559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 22:45:57,354 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 22:45:57.812578 15075 mini_ranger_kms.cc:310] Created kms service
I20260926 22:45:58.026530 15075 mini_ranger_kms.cc:326] Created kudu user
I20260926 22:45:58.067305 15075 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 22:45:58 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462755, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:45:58.451076 15075 mini_ranger_kms.cc:384] Added ranger policy
I20260926 22:45:58.451368 15075 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 22:45:58.451395 15075 mini_ranger_kms.cc:240] Using host: 127.14.184.212
I20260926 22:45:58.454743 15075 mini_ranger_kms.cc:276] Ranger KMS PID: 26184
I20260926 22:45:58.454864 15075 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.184.212:53061
Sep 26, 2026 10:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 10:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 10:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 10:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 10:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 10:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 22:46:00.670001 15075 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.105s user 0.004s sys 0.206s
I20260926 22:46:00.670085 15075 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 22:46:00.670171 15075 mini_ranger_kms.cc:401] 127.14.184.212:53061/kms/v1/keys
Sep 26 22:46:00 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790462760, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.184.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 22:46:01 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462760, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.184.212@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
Sep 26 22:46:01 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462760, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.184.212@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
Sep 26 22:46:03 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:46:03.794257 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34661
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:34661
--ranger_config_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:53061/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:03.874874 26235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:03.875100 26235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:03.875133 26235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:03.876242 26235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:03.876307 26235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:03.876329 26235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:03.876349 26235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:03.876381 26235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:03.877868 26235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.184.212:53061/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:34661
--ranger_config_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:03.878129 26235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:03.878347 26235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:03.883435 26240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:03.883486 26235 server_base.cc:1034] running on GCE node
W20260926 22:46:03.883613 26241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:03.883431 26243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:03.883941 26235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:03.884274 26235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:03.885478 26235 hybrid_clock.cc:648] HybridClock initialized: now 1790462763885453 us; error 43 us; skew 500 ppm
Sep 26 22:46:03 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462763, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:03.887913 26235 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:46:03.888298 26235 webserver.cc:460] Webserver started at http://127.14.184.254:34651/ using document root <none> and password file <none>
I20260926 22:46:03.888468 26235 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:03.888504 26235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:03.888587 26235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:03 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462763, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:03.944748 26235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "ce59cf8b96ae4f91b379048dc7a9183b"
format_stamp: "Formatted at 2026-09-26 22:46:03 on dist-test-slave-3986"
server_key: "4b6cf1ed07a5179381b20f5e18867075"
server_key_iv: "7df085c2b29a39ee9c2d06e0ec53a49f"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:03.945191 26235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "ce59cf8b96ae4f91b379048dc7a9183b"
format_stamp: "Formatted at 2026-09-26 22:46:03 on dist-test-slave-3986"
server_key: "4b6cf1ed07a5179381b20f5e18867075"
server_key_iv: "7df085c2b29a39ee9c2d06e0ec53a49f"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:03.947459 26235 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 22:46:03.958211 26250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:03.958434 26235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:03.958528 26235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "ce59cf8b96ae4f91b379048dc7a9183b"
format_stamp: "Formatted at 2026-09-26 22:46:03 on dist-test-slave-3986"
server_key: "4b6cf1ed07a5179381b20f5e18867075"
server_key_iv: "7df085c2b29a39ee9c2d06e0ec53a49f"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:03.958616 26235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:03.975778 26235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:03.977408 26235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:03.977566 26235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:03.981776 26235 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:34661
I20260926 22:46:03.981809 26302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:34661 every 8 connection(s)
I20260926 22:46:03.982172 26235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:46:03.982945 26303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:03.986063 26303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b: Bootstrap starting.
I20260926 22:46:03.986763 26303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:03.987016 26303 log.cc:824] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:03.987671 26303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b: No bootstrap required, opened a new log
I20260926 22:46:03.989542 26303 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34661 } }
I20260926 22:46:03.989672 26303 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:03.989696 26303 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce59cf8b96ae4f91b379048dc7a9183b, State: Initialized, Role: FOLLOWER
I20260926 22:46:03.989832 26303 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34661 } }
I20260926 22:46:03.989909 26303 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:46:03.989933 26303 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:46:03.990005 26303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:03.990134 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 26235
I20260926 22:46:03.990242 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:46:03.990455 15075 external_mini_cluster.cc:1428] Setting key 6146dbc72d8f3db9ab98257432ac5a5f
I20260926 22:46:03.990736 26303 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34661 } }
I20260926 22:46:03.990854 26303 leader_election.cc:302] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce59cf8b96ae4f91b379048dc7a9183b; no voters:
I20260926 22:46:03.991055 26303 leader_election.cc:288] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:46:03.991091 26307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:03.991267 26307 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [term 1 LEADER]: Becoming Leader. State: Replica: ce59cf8b96ae4f91b379048dc7a9183b, State: Running, Role: LEADER
I20260926 22:46:03.991343 26307 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34661 } }
I20260926 22:46:03.991380 26303 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:03.991616 26307 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce59cf8b96ae4f91b379048dc7a9183b. Latest consensus state: current_term: 1 leader_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34661 } } }
I20260926 22:46:03.991680 26307 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:03.991896 26312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:46:03.992401 26312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:46:03.992794 26303 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 22:46:03.994848 26309 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce59cf8b96ae4f91b379048dc7a9183b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34661 } } }
I20260926 22:46:03.995128 26309 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:03.999182 26312 catalog_manager.cc:1347] Generated new cluster ID: 291bf9aa00cd4ba1bee1c000223fd5d1
I20260926 22:46:03.999250 26312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:46:03 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:04.005198 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:03.990983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.14.184.254:34661)
0926 22:46:03.991187 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:03.991206 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:03.991242 (+ 36us) server_negotiation.cc:407] Connection header received
0926 22:46:03.991381 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:03.991388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:03.991441 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:03.991589 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:03.992465 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:03.995868 (+ 3403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:03.997070 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:03.997884 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:03.997985 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:04.000990 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:04.000993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:04.000994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:04.001005 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:04.003047 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:04.003470 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:04.003473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:04.003473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:04.003584 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:04.003801 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:04.003802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:04.003802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:04.004116 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:04.004293 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:04.004814 (+ 521us) server_negotiation.cc:299] Negotiation successful
0926 22:46:04.004948 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260926 22:46:04.010422 26312 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:46:04.011152 26312 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:46:04.022156 26312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b: Generated new TSK 0
I20260926 22:46:04.022318 26312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:46:04.681707 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:53061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34661
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:04.770776 26348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:04.771003 26348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:04.771041 26348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:04.772295 26348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:04.772357 26348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:04.772401 26348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:46:04.773908 26348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.184.212:53061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:04.774171 26348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:04.774402 26348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:04.779408 26356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:04.779408 26353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:04.779809 26354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:04.780387 26348 server_base.cc:1034] running on GCE node
I20260926 22:46:04.780593 26348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:04.780879 26348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:04.782030 26348 hybrid_clock.cc:648] HybridClock initialized: now 1790462764782011 us; error 37 us; skew 500 ppm
Sep 26 22:46:04 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:04.784719 26348 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:46:04.785202 26348 webserver.cc:460] Webserver started at http://127.14.184.193:45607/ using document root <none> and password file <none>
I20260926 22:46:04.785463 26348 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:04.785531 26348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:04.785732 26348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:04 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:04.809638 26348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "264f4a55c2634f5cba7587a95af40606"
format_stamp: "Formatted at 2026-09-26 22:46:04 on dist-test-slave-3986"
server_key: "fcaca9692db84b69a644f970ec3c8c07"
server_key_iv: "e10240507d8f045b1888f5921c2c9af7"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:04.810084 26348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "264f4a55c2634f5cba7587a95af40606"
format_stamp: "Formatted at 2026-09-26 22:46:04 on dist-test-slave-3986"
server_key: "fcaca9692db84b69a644f970ec3c8c07"
server_key_iv: "e10240507d8f045b1888f5921c2c9af7"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:04.814208 26348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 22:46:04.830822 26363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:04.831068 26348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 22:46:04.831158 26348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "264f4a55c2634f5cba7587a95af40606"
format_stamp: "Formatted at 2026-09-26 22:46:04 on dist-test-slave-3986"
server_key: "fcaca9692db84b69a644f970ec3c8c07"
server_key_iv: "e10240507d8f045b1888f5921c2c9af7"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:04.831255 26348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:04.854635 26348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:04.856817 26348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:04.856973 26348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:04.857188 26348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:04.857546 26348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:04.857599 26348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:04.857641 26348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:04.857667 26348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:04.864785 26348 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:34159
I20260926 22:46:04.864868 26476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:34159 every 8 connection(s)
I20260926 22:46:04.865288 26348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:46:04.866837 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 26348
I20260926 22:46:04.866938 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:46:04.867138 15075 external_mini_cluster.cc:1428] Setting key d6868343079261438c6ed35ac616a62d
Sep 26 22:46:04 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:04.877969 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:04.866049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:56873 (local address 127.14.184.254:34661)
0926 22:46:04.867397 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:04.867400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:04.867409 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:46:04.867436 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:04.867437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:04.867450 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:04.867524 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:04.869891 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:04.870726 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:04.872064 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:04.872287 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:04.872365 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:04.874459 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:04.874461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:04.874461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:04.874468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:04.876383 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:04.876884 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:04.876885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:04.876886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:04.876949 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:04.877205 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:04.877206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:04.877207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:04.877333 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:04.877416 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:04.877768 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 22:46:04.877855 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1315,"thread_start_us":84,"threads_started":1}
I20260926 22:46:04.878757 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:04.866006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34661 (local address 127.14.184.193:56873)
0926 22:46:04.866166 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:04.866214 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:04.866877 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:04.867615 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:04.867620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:04.869482 (+ 1862us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:04.869768 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:04.869776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:04.870837 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:04.870838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:04.871970 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:04.871972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:04.872383 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:04.872384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:04.872510 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:04.873090 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:04.873115 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:04.874366 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:04.876463 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:04.876470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:04.876472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:04.876789 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:04.877016 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:04.877017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:04.877017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:04.877139 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:04.877408 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:04.877414 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:04.877585 (+ 171us) client_negotiation.cc:769] Sending connection context
0926 22:46:04.878322 (+ 737us) client_negotiation.cc:240] Negotiation successful
0926 22:46:04.878484 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260926 22:46:04.879500 26477 heartbeater.cc:347] Connected to a master server at 127.14.184.254:34661
I20260926 22:46:04.879597 26477 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:04.879783 26477 heartbeater.cc:511] Master 127.14.184.254:34661 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
I20260926 22:46:04.880378 26267 ts_manager.cc:194] Registered new tserver with Master: 264f4a55c2634f5cba7587a95af40606 (127.14.184.193:34159)
I20260926 22:46:04.881248 26267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:56873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:46:04.901181 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:53061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34661
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:05.008868 26485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:05.009148 26485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:05.009192 26485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:05.010967 26485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:05.011029 26485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:05.011070 26485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:46:05.013290 26485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.184.212:53061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:05.013527 26485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:05.013833 26485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:05.020066 26495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:05.020160 26493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:05.020067 26492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:05.021153 26485 server_base.cc:1034] running on GCE node
I20260926 22:46:05.021370 26485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:05.021646 26485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:05.022814 26485 hybrid_clock.cc:648] HybridClock initialized: now 1790462765022788 us; error 37 us; skew 500 ppm
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:05.026085 26485 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:46:05.026618 26485 webserver.cc:460] Webserver started at http://127.14.184.194:45073/ using document root <none> and password file <none>
I20260926 22:46:05.026896 26485 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:05.026971 26485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:05.027117 26485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:05.044814 26485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "0f97298bbf6742e9b88114de32cb7a41"
format_stamp: "Formatted at 2026-09-26 22:46:05 on dist-test-slave-3986"
server_key: "5f5219dd0d0869afc528f6566ed758a7"
server_key_iv: "aca49b6b4a8b54f0e99ad6f38380591b"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:05.045478 26485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "0f97298bbf6742e9b88114de32cb7a41"
format_stamp: "Formatted at 2026-09-26 22:46:05 on dist-test-slave-3986"
server_key: "5f5219dd0d0869afc528f6566ed758a7"
server_key_iv: "aca49b6b4a8b54f0e99ad6f38380591b"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:05.047418 26485 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260926 22:46:05.060407 26502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:05.068576 26485 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260926 22:46:05.068696 26485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "0f97298bbf6742e9b88114de32cb7a41"
format_stamp: "Formatted at 2026-09-26 22:46:05 on dist-test-slave-3986"
server_key: "5f5219dd0d0869afc528f6566ed758a7"
server_key_iv: "aca49b6b4a8b54f0e99ad6f38380591b"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:05.068795 26485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:05.099179 26485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:05.101675 26485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:05.101878 26485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:05.102154 26485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:05.102583 26485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:05.102689 26485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:05.102766 26485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:05.102823 26485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:05.109180 26485 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:42881
I20260926 22:46:05.109696 26485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:46:05.111969 26615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:42881 every 8 connection(s)
I20260926 22:46:05.118949 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 26485
I20260926 22:46:05.119076 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:46:05.119310 15075 external_mini_cluster.cc:1428] Setting key 757833f727224385ef02dc7c44fd728d
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:05.124224 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.112920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:50853 (local address 127.14.184.254:34661)
0926 22:46:05.112970 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.112972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.113193 (+ 221us) server_negotiation.cc:407] Connection header received
0926 22:46:05.113879 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.113881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.113893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.113924 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:05.114784 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.115552 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.117100 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.117432 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.117479 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.120817 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:05.120818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:05.120819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:05.120826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:05.122607 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:05.123134 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:05.123136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:05.123136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:05.123197 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:05.123486 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:05.123487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:05.123487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:05.123613 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:05.123662 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.124017 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.124106 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:46:05.124985 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.112648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34661 (local address 127.14.184.194:50853)
0926 22:46:05.113067 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.113110 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.113750 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.114106 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.114110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.114356 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:05.114623 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.114629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.115678 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.115679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.116994 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.116997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.117582 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.117583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.117726 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.118376 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:05.118386 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:05.120710 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:05.122741 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:05.122748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:05.122751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:05.123042 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:05.123282 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:05.123283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:05.123283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:05.123416 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:05.123691 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:05.123696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:05.123868 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.124592 (+ 724us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.124755 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":80,"threads_started":1}
I20260926 22:46:05.125622 26616 heartbeater.cc:347] Connected to a master server at 127.14.184.254:34661
I20260926 22:46:05.125712 26616 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:05.125890 26616 heartbeater.cc:511] Master 127.14.184.254:34661 requested a full tablet report, sending...
I20260926 22:46:05.126317 26267 ts_manager.cc:194] Registered new tserver with Master: 0f97298bbf6742e9b88114de32cb7a41 (127.14.184.194:42881)
I20260926 22:46:05.126827 26267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:50853
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:46:05.165061 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:53061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34661
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:05.267974 26622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:05.268211 26622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:05.268261 26622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:05.269464 26622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:05.269538 26622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:05.269583 26622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:46:05.271184 26622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.184.212:53061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:05.271456 26622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:05.271692 26622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:05.276814 26628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:05.276844 26630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:05.277235 26622 server_base.cc:1034] running on GCE node
W20260926 22:46:05.276938 26627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:05.277508 26622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:05.277781 26622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:05.278945 26622 hybrid_clock.cc:648] HybridClock initialized: now 1790462765278917 us; error 46 us; skew 500 ppm
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:05.281836 26622 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:46:05.282378 26622 webserver.cc:460] Webserver started at http://127.14.184.195:43273/ using document root <none> and password file <none>
I20260926 22:46:05.282646 26622 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:05.282755 26622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:05.282907 26622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:05.296010 26622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "015ee21f089d458fb92cf5d70863feec"
format_stamp: "Formatted at 2026-09-26 22:46:05 on dist-test-slave-3986"
server_key: "1c268f719dca8cb2000efbae541832cc"
server_key_iv: "dd9b7b4623805060cdaf0a09d889c47b"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:05.296408 26622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "015ee21f089d458fb92cf5d70863feec"
format_stamp: "Formatted at 2026-09-26 22:46:05 on dist-test-slave-3986"
server_key: "1c268f719dca8cb2000efbae541832cc"
server_key_iv: "dd9b7b4623805060cdaf0a09d889c47b"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:05.297935 26622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:46:05.310094 26637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:05.310390 26622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 22:46:05.310550 26622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "015ee21f089d458fb92cf5d70863feec"
format_stamp: "Formatted at 2026-09-26 22:46:05 on dist-test-slave-3986"
server_key: "1c268f719dca8cb2000efbae541832cc"
server_key_iv: "dd9b7b4623805060cdaf0a09d889c47b"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:05.310740 26622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:05.345052 26622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:05.347867 26622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:05.348104 26622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:05.348551 26622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:05.349009 26622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:05.349064 26622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:05.349128 26622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:05.349157 26622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:05.362614 26622 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:39921
I20260926 22:46:05.363193 26622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:46:05.364256 26750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:39921 every 8 connection(s)
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:05.371949 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 26622
I20260926 22:46:05.372067 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:46:05.372313 15075 external_mini_cluster.cc:1428] Setting key 360ca55bb7e0a6982a24d1847e3218e6
I20260926 22:46:05.375337 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.365526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:49477 (local address 127.14.184.254:34661)
0926 22:46:05.365643 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.365646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.365781 (+ 135us) server_negotiation.cc:407] Connection header received
0926 22:46:05.366702 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.366704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.366717 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.366791 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:05.367560 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.368391 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.369472 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.369691 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.369784 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.372128 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:05.372129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:05.372130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:05.372137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:05.373930 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:05.374343 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:05.374344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:05.374344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:05.374407 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:05.374652 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:05.374653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:05.374653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:05.374807 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:05.374884 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.375159 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.375248 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 22:46:05.375541 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.364988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34661 (local address 127.14.184.195:49477)
0926 22:46:05.365627 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.365677 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.366536 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.366902 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.366907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.367171 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:05.367456 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.367463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.368504 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.368505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.369367 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.369370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.369793 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.369794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.369935 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.370609 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:05.370617 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:05.371991 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:05.374019 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:05.374026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:05.374028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:05.374259 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:05.374472 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:05.374473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:05.374473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:05.374585 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:05.374876 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:05.374882 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:05.375029 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.375142 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.375295 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":98,"threads_started":1}
I20260926 22:46:05.376026 26751 heartbeater.cc:347] Connected to a master server at 127.14.184.254:34661
I20260926 22:46:05.376117 26751 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:05.376325 26751 heartbeater.cc:511] Master 127.14.184.254:34661 requested a full tablet report, sending...
I20260926 22:46:05.376724 26267 ts_manager.cc:194] Registered new tserver with Master: 015ee21f089d458fb92cf5d70863feec (127.14.184.195:39921)
I20260926 22:46:05.377204 26267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:49477
I20260926 22:46:05.384155 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:46:05.392379 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.385703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.14.184.254:34661)
0926 22:46:05.385768 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.385770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.385903 (+ 133us) server_negotiation.cc:407] Connection header received
0926 22:46:05.385977 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.385979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.385989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.386040 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:05.386604 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.387127 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.387855 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.388062 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.388134 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.389370 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:05.389371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:05.389372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:05.389379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:05.391040 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:05.391440 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:05.391442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:05.391442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:05.391498 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:05.391699 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:05.391700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:05.391701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:05.391821 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:05.391934 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.392168 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.392265 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:46:05.394091 26267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:46:05.396981 26267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:46:05.420044 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.414915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:34159 (local address 127.0.0.1:56672)
0926 22:46:05.415351 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.415396 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.415469 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.415868 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.415871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.415873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:05.415947 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.415949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.416831 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.416833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.418467 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.418470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.419456 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.419457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.419528 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.419540 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.419606 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.419661 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":82,"threads_started":1}
I20260926 22:46:05.420817 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.414925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:39921 (local address 127.0.0.1:35934)
0926 22:46:05.416882 (+ 1957us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.416893 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.416940 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.417191 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.417192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.417194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:05.417227 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.417228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.418478 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.418479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.419554 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.419556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.420593 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.420594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.420660 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.420664 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.420710 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.420744 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1939,"thread_start_us":76,"threads_started":1}
I20260926 22:46:05.421062 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.415182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.14.184.195:39921)
0926 22:46:05.415385 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.415393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.416999 (+ 1606us) server_negotiation.cc:407] Connection header received
0926 22:46:05.417040 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.417043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.417058 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.417123 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:05.417312 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.418386 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.419657 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.420419 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.420501 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.420560 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.420938 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.420985 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":69,"threads_started":1}
I20260926 22:46:05.421780 26683 tablet_service.cc:1467] Processing CreateTablet for tablet 91615c10e85a40c3a4753e69003d3f7c (DEFAULT_TABLE table=test-table [id=c72c15da02814452b407af25ac60c571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:05.422122 26683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91615c10e85a40c3a4753e69003d3f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:05.424940 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.415175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.14.184.193:34159)
0926 22:46:05.415592 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.415596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.415605 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:46:05.415642 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.415644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.415661 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.415797 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:05.416031 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.416744 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.418549 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.419302 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.419371 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.419424 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.424798 (+ 5374us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.424849 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":52,"threads_started":1}
I20260926 22:46:05.425575 26411 tablet_service.cc:1467] Processing CreateTablet for tablet 91615c10e85a40c3a4753e69003d3f7c (DEFAULT_TABLE table=test-table [id=c72c15da02814452b407af25ac60c571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:05.425711 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.415179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.14.184.194:42881)
0926 22:46:05.415478 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.415481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.420568 (+ 5087us) server_negotiation.cc:407] Connection header received
0926 22:46:05.420609 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.420612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.420626 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.420694 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:05.421431 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.422416 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.423897 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.424585 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.424663 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.424750 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.425595 (+ 845us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.425635 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":86,"threads_started":1}
I20260926 22:46:05.425928 26411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91615c10e85a40c3a4753e69003d3f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:05.426093 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.414948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:42881 (local address 127.0.0.1:38220)
0926 22:46:05.420462 (+ 5514us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.420472 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.420509 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.420781 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.420782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.420783 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:05.420807 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.420808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.422794 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.422795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.423786 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.423788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.425188 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.425189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.425256 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.425260 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.425315 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.425350 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5503,"thread_start_us":76,"threads_started":1}
I20260926 22:46:05.426301 26769 tablet_bootstrap.cc:492] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Bootstrap starting.
I20260926 22:46:05.426716 26550 tablet_service.cc:1467] Processing CreateTablet for tablet 91615c10e85a40c3a4753e69003d3f7c (DEFAULT_TABLE table=test-table [id=c72c15da02814452b407af25ac60c571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:05.427038 26550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91615c10e85a40c3a4753e69003d3f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:05.429373 26770 tablet_bootstrap.cc:492] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606: Bootstrap starting.
I20260926 22:46:05.431573 26769 tablet_bootstrap.cc:654] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:05.431951 26769 log.cc:824] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:05.435627 26770 tablet_bootstrap.cc:654] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:05.436044 26770 log.cc:824] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:05.436381 26769 tablet_bootstrap.cc:492] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: No bootstrap required, opened a new log
I20260926 22:46:05.436443 26771 tablet_bootstrap.cc:492] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Bootstrap starting.
I20260926 22:46:05.436506 26769 ts_tablet_manager.cc:1397] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260926 22:46:05.437448 26771 tablet_bootstrap.cc:654] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:05.437799 26771 log.cc:824] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:05.438750 26769 raft_consensus.cc:348] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.438984 26770 tablet_bootstrap.cc:492] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606: No bootstrap required, opened a new log
I20260926 22:46:05.439062 26770 ts_tablet_manager.cc:1397] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260926 22:46:05.439015 26769 raft_consensus.cc:374] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:05.439188 26769 raft_consensus.cc:729] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ee21f089d458fb92cf5d70863feec, State: Initialized, Role: FOLLOWER
I20260926 22:46:05.439422 26769 consensus_queue.cc:260] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.439792 26769 ts_tablet_manager.cc:1428] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:46:05.439956 26751 heartbeater.cc:503] Master 127.14.184.254:34661 was elected leader, sending a full tablet report...
I20260926 22:46:05.441612 26770 raft_consensus.cc:348] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.441785 26770 raft_consensus.cc:374] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:05.441833 26770 raft_consensus.cc:729] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264f4a55c2634f5cba7587a95af40606, State: Initialized, Role: FOLLOWER
I20260926 22:46:05.441949 26770 consensus_queue.cc:260] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.442210 26770 ts_tablet_manager.cc:1428] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:46:05.442257 26477 heartbeater.cc:503] Master 127.14.184.254:34661 was elected leader, sending a full tablet report...
I20260926 22:46:05.442890 26771 tablet_bootstrap.cc:492] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: No bootstrap required, opened a new log
I20260926 22:46:05.442960 26771 ts_tablet_manager.cc:1397] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260926 22:46:05.444905 26771 raft_consensus.cc:348] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.445057 26771 raft_consensus.cc:374] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:05.445094 26771 raft_consensus.cc:729] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f97298bbf6742e9b88114de32cb7a41, State: Initialized, Role: FOLLOWER
I20260926 22:46:05.445200 26771 consensus_queue.cc:260] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.445441 26771 ts_tablet_manager.cc:1428] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 22:46:05.445575 26616 heartbeater.cc:503] Master 127.14.184.254:34661 was elected leader, sending a full tablet report...
Sep 26 22:46:05 dist-test-slave-3986 krb5kdc[25791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:05.574378 26777 raft_consensus.cc:482] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:46:05.574517 26777 raft_consensus.cc:504] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.574962 26777 leader_election.cc:288] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 264f4a55c2634f5cba7587a95af40606 (127.14.184.193:34159), 015ee21f089d458fb92cf5d70863feec (127.14.184.195:39921)
I20260926 22:46:05.578825 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.575248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:42839 (local address 127.14.184.195:39921)
0926 22:46:05.575296 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.575298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.575311 (+ 13us) server_negotiation.cc:407] Connection header received
0926 22:46:05.575343 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.575344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.575353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.575411 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:05.575602 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.576363 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.577797 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.578358 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.578423 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.578431 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.578715 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.578755 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:46:05.579181 26618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.575125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:39921 (local address 127.14.184.194:42839)
0926 22:46:05.575238 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.575250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.575289 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.575463 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.575465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.575466 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:05.575533 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.575535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.576476 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.576477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.577691 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.577694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.578461 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.578462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.578548 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.578552 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.578598 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.578639 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260926 22:46:05.579738 26692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91615c10e85a40c3a4753e69003d3f7c" candidate_uuid: "0f97298bbf6742e9b88114de32cb7a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ee21f089d458fb92cf5d70863feec" is_pre_election: true
I20260926 22:46:05.579913 26692 raft_consensus.cc:2454] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f97298bbf6742e9b88114de32cb7a41 in term 0.
I20260926 22:46:05.580175 26506 leader_election.cc:302] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ee21f089d458fb92cf5d70863feec, 0f97298bbf6742e9b88114de32cb7a41; no voters:
I20260926 22:46:05.580350 26777 raft_consensus.cc:2781] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:46:05.580396 26777 raft_consensus.cc:482] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:46:05.580418 26777 raft_consensus.cc:3037] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:05.581322 26777 raft_consensus.cc:504] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.581454 26777 leader_election.cc:288] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [CANDIDATE]: Term 1 election: Requested vote from peers 264f4a55c2634f5cba7587a95af40606 (127.14.184.193:34159), 015ee21f089d458fb92cf5d70863feec (127.14.184.195:39921)
I20260926 22:46:05.581697 26692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91615c10e85a40c3a4753e69003d3f7c" candidate_uuid: "0f97298bbf6742e9b88114de32cb7a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ee21f089d458fb92cf5d70863feec"
I20260926 22:46:05.581784 26692 raft_consensus.cc:3037] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:05.582655 26692 raft_consensus.cc:2454] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f97298bbf6742e9b88114de32cb7a41 in term 1.
I20260926 22:46:05.582739 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.575607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:38919 (local address 127.14.184.193:34159)
0926 22:46:05.575646 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.575648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.579299 (+ 3651us) server_negotiation.cc:407] Connection header received
0926 22:46:05.579406 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.579407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.579418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.579447 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:05.579681 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.580332 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.581700 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.582252 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.582305 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.582314 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.582600 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.582652 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:46:05.582998 26506 leader_election.cc:302] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ee21f089d458fb92cf5d70863feec, 0f97298bbf6742e9b88114de32cb7a41; no voters:
I20260926 22:46:05.583129 26777 raft_consensus.cc:2781] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:05.583278 26777 raft_consensus.cc:686] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 1 LEADER]: Becoming Leader. State: Replica: 0f97298bbf6742e9b88114de32cb7a41, State: Running, Role: LEADER
I20260926 22:46:05.583377 26777 consensus_queue.cc:237] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } }
I20260926 22:46:05.583729 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.575431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:34159 (local address 127.14.184.194:38919)
0926 22:46:05.579227 (+ 3796us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.579240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:05.579330 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:05.579512 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:05.579513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:05.579515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:05.579596 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.579598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.580433 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.580434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.581599 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:05.581601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.582440 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.582441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:05.582510 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.582515 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:46:05.583590 (+ 1075us) client_negotiation.cc:240] Negotiation successful
0926 22:46:05.583633 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3780,"thread_start_us":69,"threads_started":1}
I20260926 22:46:05.584187 26431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91615c10e85a40c3a4753e69003d3f7c" candidate_uuid: "0f97298bbf6742e9b88114de32cb7a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264f4a55c2634f5cba7587a95af40606" is_pre_election: true
I20260926 22:46:05.584340 26431 raft_consensus.cc:2454] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f97298bbf6742e9b88114de32cb7a41 in term 0.
I20260926 22:46:05.584343 26267 catalog_manager.cc:5495] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f97298bbf6742e9b88114de32cb7a41 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "0f97298bbf6742e9b88114de32cb7a41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f97298bbf6742e9b88114de32cb7a41" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260926 22:46:05.584462 26430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91615c10e85a40c3a4753e69003d3f7c" candidate_uuid: "0f97298bbf6742e9b88114de32cb7a41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264f4a55c2634f5cba7587a95af40606"
I20260926 22:46:05.584517 26430 raft_consensus.cc:3037] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:05.585161 26430 raft_consensus.cc:2454] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f97298bbf6742e9b88114de32cb7a41 in term 1.
I20260926 22:46:05.598611 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.595237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.14.184.194:42881)
0926 22:46:05.595298 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:05.595301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:05.595311 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:46:05.595399 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:05.595400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:05.595418 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:05.595480 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:46:05.595661 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.596428 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.597558 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:05.597869 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:05.597931 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:05.598121 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:46:05.598240 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:46:05.598307 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:05.598520 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 22:46:05.598537 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:46:05.600600 26692 raft_consensus.cc:1261] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 1 FOLLOWER]: Refusing update from remote peer 0f97298bbf6742e9b88114de32cb7a41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:05.600926 26430 raft_consensus.cc:1261] T 91615c10e85a40c3a4753e69003d3f7c P 264f4a55c2634f5cba7587a95af40606 [term 1 FOLLOWER]: Refusing update from remote peer 0f97298bbf6742e9b88114de32cb7a41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:05.600955 26779 consensus_queue.cc:1035] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ee21f089d458fb92cf5d70863feec" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:05.601279 26779 consensus_queue.cc:1035] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "264f4a55c2634f5cba7587a95af40606" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 34159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:05.604919 26782 mvcc.cc:205] Tried to move back new op lower bound from 7333735487897325568 to 7333735487830126592. Current Snapshot: MvccSnapshot[applied={T|T < 7333735487897325568}]
I20260926 22:46:05.606909 26784 mvcc.cc:205] Tried to move back new op lower bound from 7333735487897325568 to 7333735487830126592. Current Snapshot: MvccSnapshot[applied={T|T < 7333735487897325568}]
I20260926 22:46:05.607439 26783 mvcc.cc:205] Tried to move back new op lower bound from 7333735487897325568 to 7333735487830126592. Current Snapshot: MvccSnapshot[applied={T|T < 7333735487897325568}]
I20260926 22:46:05.609660 26266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:46:05.609751 26266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:46:05.610491 26266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce59cf8b96ae4f91b379048dc7a9183b: Sending DeleteTablet for 3 replicas of tablet 91615c10e85a40c3a4753e69003d3f7c
I20260926 22:46:05.610828 26411 tablet_service.cc:1514] Processing DeleteTablet for tablet 91615c10e85a40c3a4753e69003d3f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:05 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:56672
I20260926 22:46:05.610882 26550 tablet_service.cc:1514] Processing DeleteTablet for tablet 91615c10e85a40c3a4753e69003d3f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:05 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38220
I20260926 22:46:05.611100 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 26348
I20260926 22:46:05.611220 26683 tablet_service.cc:1514] Processing DeleteTablet for tablet 91615c10e85a40c3a4753e69003d3f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:05 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:35934
I20260926 22:46:05.616201 26793 tablet_replica.cc:330] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: stopping tablet replica
I20260926 22:46:05.616356 26793 raft_consensus.cc:2229] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:46:05.616623 26793 raft_consensus.cc:2258] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:46:05.616775 26792 tablet_replica.cc:330] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: stopping tablet replica
I20260926 22:46:05.617048 26792 raft_consensus.cc:2229] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:46:05.617443 26793 ts_tablet_manager.cc:1905] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:46:05.617621 26792 raft_consensus.cc:2258] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:46:05.622933 26792 ts_tablet_manager.cc:1905] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:46:05.624564 26793 ts_tablet_manager.cc:1918] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:46:05.626160 26792 ts_tablet_manager.cc:1918] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:46:05.626345 26793 log.cc:1196] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/91615c10e85a40c3a4753e69003d3f7c
I20260926 22:46:05.626477 26792 log.cc:1196] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/91615c10e85a40c3a4753e69003d3f7c
I20260926 22:46:05.627419 26792 ts_tablet_manager.cc:1939] T 91615c10e85a40c3a4753e69003d3f7c P 0f97298bbf6742e9b88114de32cb7a41: Deleting consensus metadata
I20260926 22:46:05.627885 26253 catalog_manager.cc:4841] TS 0f97298bbf6742e9b88114de32cb7a41 (127.14.184.194:42881): tablet 91615c10e85a40c3a4753e69003d3f7c (table test-table [id=c72c15da02814452b407af25ac60c571]) successfully deleted
I20260926 22:46:05.627148 26793 ts_tablet_manager.cc:1939] T 91615c10e85a40c3a4753e69003d3f7c P 015ee21f089d458fb92cf5d70863feec: Deleting consensus metadata
I20260926 22:46:05.628505 26254 catalog_manager.cc:4841] TS 015ee21f089d458fb92cf5d70863feec (127.14.184.195:39921): tablet 91615c10e85a40c3a4753e69003d3f7c (table test-table [id=c72c15da02814452b407af25ac60c571]) successfully deleted
W20260926 22:46:05.630050 26252 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:34159 (error 108)
I20260926 22:46:05.631520 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 26485
I20260926 22:46:05.637024 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 26622
I20260926 22:46:05.638319 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:05.637676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:34159 (local address 127.0.0.1:56674)
0926 22:46:05.638221 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:05.638272 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525}
W20260926 22:46:05.639005 26252 catalog_manager.cc:4584] TS 264f4a55c2634f5cba7587a95af40606 (127.14.184.193:34159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91615c10e85a40c3a4753e69003d3f7c: Network error: Client connection negotiation failed: client connection to 127.14.184.193:34159: connect: Connection refused (error 111)
I20260926 22:46:05.642467 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 26235
I20260926 22:46:05.647990 15075 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 22:46:05.992163 15075 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 22:46:05.992302 15075 mini_ranger.cc:66] Stopping Ranger...
I20260926 22:46:06.336340 15075 mini_ranger.cc:68] Stopped Ranger
2026-09-26 22:46:06.336 UTC [25821] LOG: received smart shutdown request
2026-09-26 22:46:06.339 UTC [25821] LOG: background worker "logical replication launcher" (PID 25830) exited with exit code 1
2026-09-26 22:46:06.340 UTC [25825] LOG: shutting down
2026-09-26 22:46:06.499 UTC [25821] LOG: database system is shut down
2026-09-26T22:46:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44731 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:46:06 dist-test-slave-3986 krb5kdc[26803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:46:06 dist-test-slave-3986 krb5kdc[26803](info): set up 2 sockets
Sep 26 22:46:06 dist-test-slave-3986 krb5kdc[26803](info): commencing operation
krb5kdc: starting...
W20260926 22:46:07.642763 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:46:07 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:46:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:46:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/rangeradmin_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/rangeradmin_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/rangerlookup_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/rangerlookup_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
I20260926 22:46:07.796783 15075 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmeYARH/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/postgres -l logfile start
2026-09-26 22:46:10.181 UTC [26833] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 22:46:10.181 UTC [26833] LOG: listening on IPv4 address "127.14.184.212", port 46703
2026-09-26 22:46:10.184 UTC [26833] LOG: listening on Unix socket "/tmp/.s.PGSQL.46703"
2026-09-26 22:46:10.219 UTC [26836] LOG: database system was shut down at 2026-09-26 22:46:08 UTC
2026-09-26 22:46:10.223 UTC [26833] LOG: database system is ready to accept connections
I20260926 22:46:11.158794 15075 mini_postgres.cc:96] Postgres bound to 46703
2026-09-26 22:46:11.164 UTC [26844] FATAL: database "slave" does not exist
127.14.184.212:46703 - accepting connections
I20260926 22:46:11.165280 15075 mini_ranger.cc:161] Starting Ranger...
I20260926 22:46:11.179811 15075 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 22:46:11.616901 15075 mini_ranger.cc:87] Created ranger Postgres database
I20260926 22:46:11.616989 15075 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-admin
2026-09-26 22:46:11,896 [I] DB FLAVOR :POSTGRES
2026-09-26 22:46:11,897 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 22:46:11,897 [I] Checking connection..
2026-09-26 22:46:11,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 22:46:12,142 [I] Checking connection passed.
2026-09-26 22:46:12,143 [I] --------- Verifying version history table ---------
2026-09-26 22:46:12,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 22:46:12,366 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 22:46:12,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 22:46:12,607 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 22:46:12,607 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 22:46:12,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 22:46:12.837 UTC [26908] WARNING: there is no transaction in progress
2026-09-26 22:46:12,843 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 22:46:12,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 22:46:13,092 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 22:46:13,092 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 22:46:13,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 22:46:13,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 22:46:13,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3986.c.gcp-upstream.internal','N') ;"
2026-09-26 22:46:13,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 22:46:14,018 [I] Table x_portal_user does not exist in database ranger
2026-09-26 22:46:14,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 22:46:14,265 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 22:46:14,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 22:46:14,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 22:46:14,741 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 22:46:14,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 22:46:15.654 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.672 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.687 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.701 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.716 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.736 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.787 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.793 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.808 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.817 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.827 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.839 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.848 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.854 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.861 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:15.867 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 22:46:16,468 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 22:46:16,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 22:46:16,710 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 22:46:16,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 22:46:16,963 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 22:46:16,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 22:46:17,222 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 22:46:17,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 22:46:17,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 22:46:17,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3986.c.gcp-upstream.internal';"
2026-09-26 22:46:17,902 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 22:46:17,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 22:46:18,154 [I] DB_PATCHES have already been applied
I20260926 22:46:18.159734 15075 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmeYARH/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 22:46:18.159808 15075 mini_ranger.cc:193] Using host: 127.14.184.212
I20260926 22:46:18.162760 15075 mini_ranger.cc:227] Ranger admin URL: http://127.14.184.212:51311
Sep 26, 2026 10:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 10:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 10:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 10:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 10:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 10:46:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 10:46:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 22:46:19.264829 15075 mini_ranger.cc:160] Time spent starting Ranger: real 8.100s user 0.001s sys 0.007s
Sep 26, 2026 10:46:39 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 10:46:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 10:46:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 10:46:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 10:46:40 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 22:46:41.070714 15075 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.184.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.184.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/rangerkms_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.184.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/rangerkms_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.184.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.184.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.184.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/HTTP_127.14.184.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 22:46:41 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462801, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 22:46:41.152617 15075 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 22:46:41.161471 15075 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 22:46:41.932240 15075 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 22:46:41.932325 15075 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms
I20260926 22:46:41.932348 15075 mini_ranger_kms.cc:227] Using postgres at 127.14.184.212:46703
2026-09-26 22:46:42,100 [I] DB FLAVOR :POSTGRES
2026-09-26 22:46:42,101 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 22:46:42,101 [I] Checking connection
2026-09-26 22:46:42,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 22:46:42,333 [I] connection success
2026-09-26 22:46:42,333 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 22:46:42,333 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 22:46:42,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 22:46:42,562 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 22:46:42,562 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 22:46:42,562 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 22:46:42,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.184.212:46703/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 22:46:42,817 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 22:46:43.230526 15075 mini_ranger_kms.cc:310] Created kms service
I20260926 22:46:43.453683 15075 mini_ranger_kms.cc:326] Created kudu user
I20260926 22:46:43.495471 15075 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 22:46:43 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462801, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:43.901238 15075 mini_ranger_kms.cc:384] Added ranger policy
I20260926 22:46:43.901348 15075 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmeYARH/build/release/bin/postgresql.jar:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmeYARH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 22:46:43.901369 15075 mini_ranger_kms.cc:240] Using host: 127.14.184.212
I20260926 22:46:43.904603 15075 mini_ranger_kms.cc:276] Ranger KMS PID: 27196
I20260926 22:46:43.904682 15075 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.184.212:35137
Sep 26, 2026 10:46:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 10:46:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 10:46:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 10:46:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 10:46:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 10:46:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 22:46:46.151356 15075 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.999s user 0.003s sys 0.149s
I20260926 22:46:46.151461 15075 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 22:46:46.151515 15075 mini_ranger_kms.cc:401] 127.14.184.212:35137/kms/v1/keys
Sep 26 22:46:46 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790462806, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.184.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 22:46:46 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462806, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.184.212@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
Sep 26 22:46:46 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462806, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.184.212@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
Sep 26 22:46:48 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:46:49.004922 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33953
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:33953
--ranger_config_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:49.084615 27247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:49.084810 27247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:49.084851 27247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:49.086022 27247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:49.086066 27247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:49.086086 27247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:49.086135 27247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:49.086161 27247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:49.087762 27247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.184.212:35137/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:33953
--ranger_config_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:49.088014 27247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:49.088233 27247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:46:49.093396 27247 server_base.cc:1034] running on GCE node
W20260926 22:46:49.093537 27253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:49.093475 27252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:49.093542 27255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:49.094049 27247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:49.094331 27247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:49.095487 27247 hybrid_clock.cc:648] HybridClock initialized: now 1790462809095472 us; error 28 us; skew 500 ppm
Sep 26 22:46:49 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462809, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:49.101104 27247 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:46:49.101522 27247 webserver.cc:460] Webserver started at http://127.14.184.254:44677/ using document root <none> and password file <none>
I20260926 22:46:49.101737 27247 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:49.101809 27247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:49.101963 27247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:49 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462809, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:49.146376 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "5d8709dc1ce5485cb74d31055bfcf4a2"
format_stamp: "Formatted at 2026-09-26 22:46:49 on dist-test-slave-3986"
server_key: "b470f9df490926b285250d6666215196"
server_key_iv: "d4532d389371db55bf7ea174790a498e"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:49.146934 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "5d8709dc1ce5485cb74d31055bfcf4a2"
format_stamp: "Formatted at 2026-09-26 22:46:49 on dist-test-slave-3986"
server_key: "b470f9df490926b285250d6666215196"
server_key_iv: "d4532d389371db55bf7ea174790a498e"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:49.148684 27247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:49.160681 27262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:49.160872 27247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 22:46:49.160995 27247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "5d8709dc1ce5485cb74d31055bfcf4a2"
format_stamp: "Formatted at 2026-09-26 22:46:49 on dist-test-slave-3986"
server_key: "b470f9df490926b285250d6666215196"
server_key_iv: "d4532d389371db55bf7ea174790a498e"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:49.161111 27247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:49.168099 27247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:49.170022 27247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:49.170171 27247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:49.174360 27247 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:33953
I20260926 22:46:49.174397 27314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:33953 every 8 connection(s)
I20260926 22:46:49.174816 27247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:46:49.175616 27315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:49.178768 27315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Bootstrap starting.
I20260926 22:46:49.179481 27315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:49.179811 27315 log.cc:824] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:49.180238 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 27247
I20260926 22:46:49.180352 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:46:49.180539 15075 external_mini_cluster.cc:1428] Setting key 9e5ad3f563230c98af0f274c4c0b7bbc
I20260926 22:46:49.180656 27315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: No bootstrap required, opened a new log
I20260926 22:46:49.183647 27315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:49.183827 27315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:49.183864 27315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8709dc1ce5485cb74d31055bfcf4a2, State: Initialized, Role: FOLLOWER
I20260926 22:46:49.184002 27315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:49.184079 27315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:46:49.184110 27315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:46:49.184188 27315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:49.184852 27315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:49.184971 27315 leader_election.cc:302] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d8709dc1ce5485cb74d31055bfcf4a2; no voters:
I20260926 22:46:49.185132 27315 leader_election.cc:288] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:46:49.185181 27320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:49.185405 27320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d8709dc1ce5485cb74d31055bfcf4a2, State: Running, Role: LEADER
I20260926 22:46:49.185494 27315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:49.185530 27320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:49.185865 27322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } } }
I20260926 22:46:49.185959 27322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:49.186712 27324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:46:49.187165 27324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:46:49.185855 27320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d8709dc1ce5485cb74d31055bfcf4a2. Latest consensus state: current_term: 1 leader_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } } }
I20260926 22:46:49.187533 27320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: This master's current role is: LEADER
Sep 26 22:46:49 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:49.188722 27324 catalog_manager.cc:1347] Generated new cluster ID: 86fbc715f64b4ccf88de5d31852ce0bd
I20260926 22:46:49.188779 27324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:46:49.189405 27315 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 22:46:49.196619 27319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:49.181178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.14.184.254:33953)
0926 22:46:49.181342 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:49.181350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:49.181386 (+ 36us) server_negotiation.cc:407] Connection header received
0926 22:46:49.182005 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:49.182011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:49.182060 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:49.182207 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:49.182888 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:49.183772 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:49.184856 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:49.185535 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:49.185947 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:49.188654 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:49.188657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:49.188658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:49.188669 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:49.193780 (+ 5111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:49.194307 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:49.194310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:49.194311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:49.194509 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:49.194808 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:49.194809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:49.194809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:49.195296 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:49.195906 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:49.195984 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 22:46:49.196266 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260926 22:46:49.224757 27324 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:46:49.225504 27324 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:46:49.234133 27324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Generated new TSK 0
I20260926 22:46:49.234316 27324 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:46:49.893565 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33953
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:49.997743 27360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:49.997931 27360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:49.997954 27360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:49.999155 27360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:49.999197 27360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:49.999254 27360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:46:50.000788 27360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.184.212:35137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:50.001044 27360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:50.001283 27360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:50.006307 27368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.006325 27365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.006388 27366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.006556 27360 server_base.cc:1034] running on GCE node
I20260926 22:46:50.006896 27360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:50.007169 27360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:50.008327 27360 hybrid_clock.cc:648] HybridClock initialized: now 1790462810008312 us; error 36 us; skew 500 ppm
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:50.011577 27360 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:46:50.012135 27360 webserver.cc:460] Webserver started at http://127.14.184.193:33675/ using document root <none> and password file <none>
I20260926 22:46:50.012449 27360 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:50.012522 27360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:50.012662 27360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:50.026221 27360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "710f32907ecc4ba3ad7bb783ab920511"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "c97c4b14a76e03700e116469c5f83a40"
server_key_iv: "6e7993b0958b4bdc5b7086a154b76e38"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.026722 27360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "710f32907ecc4ba3ad7bb783ab920511"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "c97c4b14a76e03700e116469c5f83a40"
server_key_iv: "6e7993b0958b4bdc5b7086a154b76e38"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.029668 27360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260926 22:46:50.041952 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.042158 27360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:50.042253 27360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "710f32907ecc4ba3ad7bb783ab920511"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "c97c4b14a76e03700e116469c5f83a40"
server_key_iv: "6e7993b0958b4bdc5b7086a154b76e38"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.042363 27360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:50.057595 27360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:50.059729 27360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:50.059912 27360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:50.060195 27360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:50.060565 27360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:50.060634 27360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.060685 27360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:50.060724 27360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.066867 27360 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:45399
I20260926 22:46:50.067334 27360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:46:50.072412 27490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:45399 every 8 connection(s)
I20260926 22:46:50.077524 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 27360
I20260926 22:46:50.077633 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:46:50.077855 15075 external_mini_cluster.cc:1428] Setting key e356613e8d44295a243b4e43efd2106a
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:50.083794 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.073052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33953 (local address 127.14.184.193:41173)
0926 22:46:50.073254 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:50.073294 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:50.073818 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:50.074123 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:50.074126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:50.074314 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:50.074624 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:50.074630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.075733 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.075734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:50.076996 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:50.076999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.077493 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.077494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:50.077646 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:50.078261 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:50.078272 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:50.079949 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:50.082084 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:50.082091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:50.082093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:50.082391 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:50.082620 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:50.082620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:50.082620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:50.082798 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:50.083126 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:50.083132 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:50.083295 (+ 163us) client_negotiation.cc:769] Sending connection context
0926 22:46:50.083396 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:46:50.083560 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260926 22:46:50.084236 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.073057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:41173 (local address 127.14.184.254:33953)
0926 22:46:50.073195 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:50.073197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:50.073406 (+ 209us) server_negotiation.cc:407] Connection header received
0926 22:46:50.073938 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:50.073939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:50.073955 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:50.074046 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:50.074812 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.075621 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.077109 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.077336 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.077403 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:50.080062 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:50.080064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:50.080064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:50.080073 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:50.081987 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:50.082484 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:50.082485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:50.082486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:50.082549 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:50.082879 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:50.082880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:50.082880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:50.083035 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:50.083098 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:50.084049 (+ 951us) server_negotiation.cc:299] Negotiation successful
0926 22:46:50.084137 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260926 22:46:50.084640 27491 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33953
I20260926 22:46:50.084759 27491 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:50.084944 27491 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:50.085496 27279 ts_manager.cc:194] Registered new tserver with Master: 710f32907ecc4ba3ad7bb783ab920511 (127.14.184.193:45399)
I20260926 22:46:50.086360 27279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:41173
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:46:50.109831 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33953
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:50.216187 27497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:50.216401 27497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:50.216439 27497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:50.217871 27497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:50.217942 27497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:50.217986 27497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:46:50.219936 27497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.184.212:35137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:50.220187 27497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:50.220420 27497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:50.225880 27507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.225948 27504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.226178 27497 server_base.cc:1034] running on GCE node
W20260926 22:46:50.226308 27505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.226524 27497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:50.226835 27497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:50.227998 27497 hybrid_clock.cc:648] HybridClock initialized: now 1790462810227973 us; error 38 us; skew 500 ppm
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:50.230504 27497 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:46:50.230974 27497 webserver.cc:460] Webserver started at http://127.14.184.194:37623/ using document root <none> and password file <none>
I20260926 22:46:50.231220 27497 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:50.231287 27497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:50.231457 27497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:50.246331 27497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "0a009a398a0b491786fc5aa80ce11308"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "4a5563c64619ed5bc78c5968ddefee47"
server_key_iv: "12c3e19a589e541b84e69ff85d6d9108"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.248835 27497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "0a009a398a0b491786fc5aa80ce11308"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "4a5563c64619ed5bc78c5968ddefee47"
server_key_iv: "12c3e19a589e541b84e69ff85d6d9108"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.255089 27497 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260926 22:46:50.283341 27514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.283645 27497 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.000s
I20260926 22:46:50.283768 27497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "0a009a398a0b491786fc5aa80ce11308"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "4a5563c64619ed5bc78c5968ddefee47"
server_key_iv: "12c3e19a589e541b84e69ff85d6d9108"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.283877 27497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:50.328305 27497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:50.330900 27497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:50.331108 27497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:50.331413 27497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:50.331826 27497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:50.331914 27497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.331991 27497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:50.332059 27497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.338426 27497 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:38429
I20260926 22:46:50.338943 27497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:46:50.340883 27628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:38429 every 8 connection(s)
I20260926 22:46:50.340966 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 27497
I20260926 22:46:50.341073 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:46:50.341259 15075 external_mini_cluster.cc:1428] Setting key 607f49ec6c33c771eda67342f7c5c46d
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
I20260926 22:46:50.351989 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.341864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:55829 (local address 127.14.184.254:33953)
0926 22:46:50.341923 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:50.341926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:50.342076 (+ 150us) server_negotiation.cc:407] Connection header received
0926 22:46:50.342589 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:50.342590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:50.342604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:50.342649 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:50.343237 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.344039 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.345535 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.345860 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.345905 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:50.348634 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:50.348635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:50.348636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:50.348642 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:50.350307 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:50.350837 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:50.350838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:50.350838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:50.350895 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:50.351199 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:50.351200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:50.351201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:50.351346 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:50.351405 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:50.351796 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 22:46:50.351888 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:46:50.352510 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.341609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33953 (local address 127.14.184.194:55829)
0926 22:46:50.341962 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:50.342010 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:50.342488 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:50.342737 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:50.342741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:50.342941 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:50.343135 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:50.343140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.344151 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.344152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:50.345434 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:50.345437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.346007 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.346008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:50.346156 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:50.346783 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:50.346793 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:50.348521 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:50.350409 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:50.350416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:50.350418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:50.350741 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:50.350974 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:50.350974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:50.350975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:50.351108 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:50.351443 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:50.351450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:50.351614 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 22:46:50.352125 (+ 511us) client_negotiation.cc:240] Negotiation successful
0926 22:46:50.352282 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":82,"threads_started":1}
I20260926 22:46:50.352993 27629 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33953
I20260926 22:46:50.353096 27629 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:50.353286 27629 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:50.353789 27279 ts_manager.cc:194] Registered new tserver with Master: 0a009a398a0b491786fc5aa80ce11308 (127.14.184.194:38429)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
I20260926 22:46:50.354256 27279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:55829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:46:50.370446 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33953
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:50.484153 27634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:50.484472 27634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:50.484529 27634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:50.486227 27634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:50.486289 27634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:50.486330 27634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:46:50.488489 27634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.184.212:35137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:50.488730 27634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:50.489042 27634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:50.495059 27640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.495066 27639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.495283 27642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.495985 27634 server_base.cc:1034] running on GCE node
I20260926 22:46:50.496191 27634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:50.496443 27634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:50.497619 27634 hybrid_clock.cc:648] HybridClock initialized: now 1790462810497599 us; error 34 us; skew 500 ppm
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:50.500738 27634 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:46:50.501258 27634 webserver.cc:460] Webserver started at http://127.14.184.195:45113/ using document root <none> and password file <none>
I20260926 22:46:50.501523 27634 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:50.501585 27634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:50.501760 27634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:50.529238 27634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "0b66247694594d9caed0a66b1b1388ca"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "490a7c25f6a51ad4435ca4034c88e665"
server_key_iv: "43e52640781fb3b007a57ce5b82ba96a"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.529706 27634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "0b66247694594d9caed0a66b1b1388ca"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "490a7c25f6a51ad4435ca4034c88e665"
server_key_iv: "43e52640781fb3b007a57ce5b82ba96a"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.531675 27634 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 22:46:50.553855 27649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.554121 27634 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 22:46:50.554221 27634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "0b66247694594d9caed0a66b1b1388ca"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "490a7c25f6a51ad4435ca4034c88e665"
server_key_iv: "43e52640781fb3b007a57ce5b82ba96a"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.554313 27634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:50.572949 27634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:50.575484 27634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:50.575702 27634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:50.575984 27634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:50.578397 27634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:50.578505 27634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.578583 27634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:50.578646 27634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.585115 27634 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:36727
I20260926 22:46:50.587045 27634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:46:50.588188 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 27634
I20260926 22:46:50.588303 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:46:50.588521 15075 external_mini_cluster.cc:1428] Setting key 6320560fdc8f30fe69768e2966a2cc4f
I20260926 22:46:50.596504 27762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:36727 every 8 connection(s)
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:50.610291 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.597529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:49393 (local address 127.14.184.254:33953)
0926 22:46:50.597575 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:50.597577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:50.598948 (+ 1371us) server_negotiation.cc:407] Connection header received
0926 22:46:50.600288 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:50.600289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:50.600301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:50.600332 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:50.601215 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.602005 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.603563 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.603902 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.603964 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:50.606811 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:50.606812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:50.606813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:50.606820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:50.608603 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:50.609148 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:50.609149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:50.609149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:50.609225 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:50.609513 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:50.609513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:50.609514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:50.609644 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:50.609702 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:50.610085 (+ 383us) server_negotiation.cc:299] Negotiation successful
0926 22:46:50.610171 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:46:50.610414 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.597239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33953 (local address 127.14.184.195:49393)
0926 22:46:50.598796 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:50.598849 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:50.600176 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:50.600528 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:50.600533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:50.600791 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:50.601097 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:50.601104 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.602125 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.602126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:50.603451 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:50.603454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:50.604075 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:50.604076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:50.604220 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:50.604867 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:50.604877 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:50.606704 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:50.608720 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:50.608727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:50.608729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:50.609040 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:50.609302 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:50.609303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:50.609303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:50.609436 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:50.609737 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:50.609743 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:50.609901 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 22:46:50.610012 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 22:46:50.610169 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1509,"thread_start_us":98,"threads_started":1}
I20260926 22:46:50.611485 27763 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33953
I20260926 22:46:50.611580 27763 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:50.611758 27763 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:50.612181 27279 ts_manager.cc:194] Registered new tserver with Master: 0b66247694594d9caed0a66b1b1388ca (127.14.184.195:36727)
I20260926 22:46:50.612665 27279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:49393
I20260926 22:46:50.623629 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:46:50.624413 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 27360
I20260926 22:46:50.631244 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 27497
I20260926 22:46:50.635700 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 27634
I20260926 22:46:50.640516 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 27247
I20260926 22:46:50.645742 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33953
--webserver_interface=127.14.184.254
--webserver_port=44677
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:33953
--ranger_config_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:50.725550 27766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:50.725786 27766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:50.725821 27766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:50.726992 27766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:50.727056 27766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:50.727079 27766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:50.727101 27766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:50.727128 27766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:50.728647 27766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:33953
--ranger_config_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:50.728928 27766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:50.729130 27766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:46:50.734057 27766 server_base.cc:1034] running on GCE node
W20260926 22:46:50.734002 27774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.734009 27772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.734045 27771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.734560 27766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:50.734869 27766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:50.736037 27766 hybrid_clock.cc:648] HybridClock initialized: now 1790462810736027 us; error 40 us; skew 500 ppm
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:50.738407 27766 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:46:50.738919 27766 webserver.cc:460] Webserver started at http://127.14.184.254:44677/ using document root <none> and password file <none>
I20260926 22:46:50.739128 27766 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:50.739199 27766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:50.796635 27766 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:50.797487 27782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.797713 27766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:50.797833 27766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "5d8709dc1ce5485cb74d31055bfcf4a2"
format_stamp: "Formatted at 2026-09-26 22:46:49 on dist-test-slave-3986"
server_key: "b470f9df490926b285250d6666215196"
server_key_iv: "d4532d389371db55bf7ea174790a498e"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.798225 27766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:50.812252 27766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:50.813845 27766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:50.814008 27766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:50.818228 27766 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:33953
I20260926 22:46:50.818269 27834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:33953 every 8 connection(s)
I20260926 22:46:50.818636 27766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:46:50.819847 27835 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:46:50.820508 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Bootstrap starting.
I20260926 22:46:50.821599 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 27766
I20260926 22:46:50.821866 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:45399
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:33953
--webserver_port=33675
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:46:50.828547 27835 log.cc:824] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:50.830193 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:46:50.830487 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Bootstrap complete.
I20260926 22:46:50.832865 27835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:50.833096 27835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8709dc1ce5485cb74d31055bfcf4a2, State: Initialized, Role: FOLLOWER
I20260926 22:46:50.833263 27835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:50.833338 27835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:46:50.833386 27835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:46:50.833433 27835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:46:50.834405 27835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:50.834517 27835 leader_election.cc:302] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d8709dc1ce5485cb74d31055bfcf4a2; no voters:
I20260926 22:46:50.834750 27835 leader_election.cc:288] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 22:46:50.834784 27839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:46:50.834923 27839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [term 2 LEADER]: Becoming Leader. State: Replica: 5d8709dc1ce5485cb74d31055bfcf4a2, State: Running, Role: LEADER
I20260926 22:46:50.835028 27835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:50.835106 27839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } }
I20260926 22:46:50.835482 27840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } } }
I20260926 22:46:50.835570 27840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:50.835788 27841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d8709dc1ce5485cb74d31055bfcf4a2. Latest consensus state: current_term: 2 leader_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8709dc1ce5485cb74d31055bfcf4a2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33953 } } }
I20260926 22:46:50.835861 27841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:50.836067 27847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:46:50.836611 27847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:46:50.836803 27847 catalog_manager.cc:1259] Loaded cluster ID: 86fbc715f64b4ccf88de5d31852ce0bd
I20260926 22:46:50.836889 27847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:46:50.836935 27835 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 22:46:50.840651 27847 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:46:50.841183 27847 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Loaded TSK: 0
I20260926 22:46:50.841490 27847 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 22:46:50.927811 27837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:50.928099 27837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:50.928139 27837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:50.929327 27837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:50.929386 27837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:50.929430 27837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:46:50.931114 27837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:45399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=33675
--tserver_master_addrs=127.14.184.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:50.931357 27837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:50.931574 27837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:50.936455 27875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:50.936708 27873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.939911 27837 server_base.cc:1034] running on GCE node
W20260926 22:46:50.940881 27872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:50.941181 27837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:50.941459 27837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:50.942600 27837 hybrid_clock.cc:648] HybridClock initialized: now 1790462810942568 us; error 46 us; skew 500 ppm
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:50.945474 27837 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:46:50.945971 27837 webserver.cc:460] Webserver started at http://127.14.184.193:33675/ using document root <none> and password file <none>
I20260926 22:46:50.946229 27837 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:50.946310 27837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 22:46:50 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:50.966082 27837 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:50.969061 27883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.969249 27837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 22:46:50.969324 27837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "710f32907ecc4ba3ad7bb783ab920511"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "c97c4b14a76e03700e116469c5f83a40"
server_key_iv: "6e7993b0958b4bdc5b7086a154b76e38"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:50.969723 27837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:50.985217 27837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:50.987581 27837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:50.987777 27837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:50.988026 27837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:50.988392 27837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:50.988476 27837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.988533 27837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:50.988566 27837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:50.996385 27837 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:45399
I20260926 22:46:50.996793 27837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:46:50.997777 27996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:45399 every 8 connection(s)
I20260926 22:46:50.999092 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 27837
I20260926 22:46:50.999366 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:38429
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:33953
--webserver_port=37623
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462810, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:51.011634 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.998381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33953 (local address 127.14.184.193:46151)
0926 22:46:50.998742 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:50.998787 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:50.999415 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:51.000071 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:51.000075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:51.000325 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:51.000592 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:51.000599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.002285 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.002286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:51.003747 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:51.003750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.004589 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.004591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:51.004759 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.005352 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:51.005362 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:51.007167 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:51.009683 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:51.009690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:51.009692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:51.010014 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:51.010315 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:51.010315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:51.010316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:51.010469 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:51.011005 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:51.011010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:51.011162 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 22:46:51.011253 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 22:46:51.011395 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":76,"threads_started":1}
I20260926 22:46:51.012245 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:50.998474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:46151 (local address 127.14.184.254:33953)
0926 22:46:50.998802 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:50.998812 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:50.998882 (+ 70us) server_negotiation.cc:407] Connection header received
0926 22:46:50.999715 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:50.999722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:50.999780 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:50.999913 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:51.000766 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.002136 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.003858 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.004465 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.004587 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.007302 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:51.007306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:51.007306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:51.007321 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:51.009582 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.010117 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.010120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.010121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.010235 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.010554 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.010555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.010555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.010895 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:51.010975 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:51.011936 (+ 961us) server_negotiation.cc:299] Negotiation successful
0926 22:46:51.012052 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":86,"threads_started":1}
I20260926 22:46:51.012849 27997 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33953
I20260926 22:46:51.012945 27997 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.013126 27997 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
W20260926 22:46:51.014030 27997 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:33953 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 22:46:51.014101 27997 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.014158 27997 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:51.014451 27997 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.014520 27997 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
W20260926 22:46:51.014834 27997 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 22:46:51.105947 28001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:51.106247 28001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:51.106323 28001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:51.107545 28001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:51.107610 28001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:51.107654 28001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:46:51.109205 28001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:38429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=37623
--tserver_master_addrs=127.14.184.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:51.109454 28001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:51.109687 28001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:51.115504 28009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:51.115513 28007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:51.115854 28006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:51.115998 28001 server_base.cc:1034] running on GCE node
I20260926 22:46:51.116173 28001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:51.116422 28001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:51.117579 28001 hybrid_clock.cc:648] HybridClock initialized: now 1790462811117546 us; error 45 us; skew 500 ppm
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:51.120575 28001 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:46:51.121053 28001 webserver.cc:460] Webserver started at http://127.14.184.194:37623/ using document root <none> and password file <none>
I20260926 22:46:51.121297 28001 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:51.121356 28001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:51.146158 28001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:51.147629 28017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:51.147818 28001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 22:46:51.147910 28001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "0a009a398a0b491786fc5aa80ce11308"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "4a5563c64619ed5bc78c5968ddefee47"
server_key_iv: "12c3e19a589e541b84e69ff85d6d9108"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:51.148301 28001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:51.160998 28001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:51.163445 28001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:51.163601 28001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:51.163846 28001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:51.164238 28001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:51.164304 28001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:51.164348 28001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:51.164381 28001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:51.170476 28001 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:38429
I20260926 22:46:51.170647 28130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:38429 every 8 connection(s)
I20260926 22:46:51.172291 28001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:46:51.175501 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28001
I20260926 22:46:51.175791 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.184.212:35137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:36727
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:33953
--webserver_port=45113
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:51.182592 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:51.173099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:49923 (local address 127.14.184.254:33953)
0926 22:46:51.173150 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:51.173152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:51.173443 (+ 291us) server_negotiation.cc:407] Connection header received
0926 22:46:51.174006 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:51.174008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:51.174023 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:51.174080 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:51.174792 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.175288 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.176832 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.177170 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.177215 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.179836 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:51.179837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:51.179838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:51.179845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:51.181249 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.181672 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.181673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.181674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.181731 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.181973 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.181974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.181974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.182088 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:51.182156 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:51.182422 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 22:46:51.182505 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:46:51.182796 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:51.173100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33953 (local address 127.14.184.194:49923)
0926 22:46:51.173318 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:51.173355 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:51.173900 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:51.174153 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:51.174157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:51.174415 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:51.174652 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:51.174657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.175422 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.175424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:51.176714 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:51.176717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.177329 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.177331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:51.177483 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.178071 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:51.178081 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:51.179709 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:51.181364 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:51.181371 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:51.181372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:51.181586 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:51.181799 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:51.181800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:51.181800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:51.181907 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:51.182158 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:51.182165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:51.182298 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 22:46:51.182403 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 22:46:51.182537 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260926 22:46:51.183346 28131 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33953
I20260926 22:46:51.183449 28131 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.183646 28131 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
W20260926 22:46:51.184237 28131 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:33953 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 22:46:51.184322 28131 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.184393 28131 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:51.184664 28131 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.184746 28131 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
W20260926 22:46:51.185006 28131 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 22:46:51.301316 28134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:51.301577 28134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:51.301612 28134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:51.303316 28134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:51.303373 28134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:51.303412 28134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:46:51.305481 28134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.184.212:35137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:36727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=45113
--tserver_master_addrs=127.14.184.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:51.305889 28134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:51.306174 28134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:51.312809 28139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:51.312857 28140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:51.312799 28142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:51.313601 28134 server_base.cc:1034] running on GCE node
I20260926 22:46:51.313823 28134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:51.314095 28134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:51.315281 28134 hybrid_clock.cc:648] HybridClock initialized: now 1790462811315251 us; error 47 us; skew 500 ppm
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:51.318408 28134 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:46:51.318930 28134 webserver.cc:460] Webserver started at http://127.14.184.195:45113/ using document root <none> and password file <none>
I20260926 22:46:51.319185 28134 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:51.319283 28134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:51.344477 28134 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:51.345969 28151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:51.346222 28134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:46:51.346359 28134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "0b66247694594d9caed0a66b1b1388ca"
format_stamp: "Formatted at 2026-09-26 22:46:50 on dist-test-slave-3986"
server_key: "490a7c25f6a51ad4435ca4034c88e665"
server_key_iv: "43e52640781fb3b007a57ce5b82ba96a"
server_key_version: "kuduclusterkey@0"
I20260926 22:46:51.346879 28134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:51.365917 28134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:51.368369 28134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:51.368542 28134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:51.368855 28134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:51.369270 28134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:51.369329 28134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:51.369402 28134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:51.369434 28134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:51.376783 28134 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:36727
I20260926 22:46:51.376865 28264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:36727 every 8 connection(s)
I20260926 22:46:51.377249 28134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:46:51.378249 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28134
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:51.391407 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:51.378090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:55171 (local address 127.14.184.254:33953)
0926 22:46:51.378145 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:51.378148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:51.378564 (+ 416us) server_negotiation.cc:407] Connection header received
0926 22:46:51.382834 (+ 4270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:51.382836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:51.382848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:51.382902 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:51.383451 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.384205 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.385570 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.385779 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.385863 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.388156 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:51.388158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:51.388158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:51.388165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:51.389774 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.390283 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.390284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.390284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.390346 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.390653 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.390654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.390655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.390822 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:51.390889 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:51.391243 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 22:46:51.391322 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 22:46:51.392118 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:51.378054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33953 (local address 127.14.184.195:55171)
0926 22:46:51.378392 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:51.378436 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:51.382629 (+ 4193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:51.382996 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:51.383000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:51.383167 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:51.383359 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:51.383364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.384314 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.384316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:51.385463 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:51.385466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.385892 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.385893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:51.386047 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.386609 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:51.386623 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:51.388060 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:51.389866 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:51.389872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:51.389875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:51.390173 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:51.390424 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:51.390424 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:51.390425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:51.390555 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:51.390918 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:51.390924 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:51.391075 (+ 151us) client_negotiation.cc:769] Sending connection context
0926 22:46:51.391746 (+ 671us) client_negotiation.cc:240] Negotiation successful
0926 22:46:51.391902 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":79,"threads_started":1}
I20260926 22:46:51.392144 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:51.378657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.14.184.254:33953)
0926 22:46:51.378783 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:51.378785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:51.382875 (+ 4090us) server_negotiation.cc:407] Connection header received
0926 22:46:51.383455 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:51.383456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:51.383466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:51.383554 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:51.384135 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.384871 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.385964 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:51.386284 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:51.386347 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:51.388004 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:51.388006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:51.388006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:51.388013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:51.390541 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.390960 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.390961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.390961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.391016 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:51.391223 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:51.391224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:51.391225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:51.391343 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:51.391509 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:51.391950 (+ 441us) server_negotiation.cc:299] Negotiation successful
0926 22:46:51.392028 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":90,"threads_started":1}
I20260926 22:46:51.392867 28265 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33953
I20260926 22:46:51.392982 28265 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.393162 28265 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
W20260926 22:46:51.393754 28265 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:33953 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 22:46:51.393824 28265 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.393884 28265 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:51.394236 28265 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:51.394310 28265 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
W20260926 22:46:51.394645 28265 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 22:46:51 dist-test-slave-3986 krb5kdc[26803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:52.015022 27997 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:52.015180 27997 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:52.015630 27798 ts_manager.cc:194] Registered new tserver with Master: 710f32907ecc4ba3ad7bb783ab920511 (127.14.184.193:45399)
I20260926 22:46:52.016669 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:46151
I20260926 22:46:52.185184 28131 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:52.185328 28131 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:52.185616 27798 ts_manager.cc:194] Registered new tserver with Master: 0a009a398a0b491786fc5aa80ce11308 (127.14.184.194:38429)
I20260926 22:46:52.186141 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:49923
Sep 26 22:46:52 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
I20260926 22:46:52.394855 28265 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:52.395004 28265 heartbeater.cc:511] Master 127.14.184.254:33953 requested a full tablet report, sending...
I20260926 22:46:52.395305 27798 ts_manager.cc:194] Registered new tserver with Master: 0b66247694594d9caed0a66b1b1388ca (127.14.184.195:36727)
I20260926 22:46:52.395840 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:55171
I20260926 22:46:52.397300 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:46:52.417680 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.398256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.14.184.254:33953)
0926 22:46:52.409916 (+ 11660us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.409918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.409929 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:46:52.409954 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.409955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.409967 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.410054 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:52.410786 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.411532 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.412609 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.412934 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.413009 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.414612 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:52.414613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:52.414613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:52.414621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:52.416442 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:52.416854 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:52.416856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:52.416856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:52.416918 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:52.417140 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:52.417141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:52.417141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:52.417265 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:52.417358 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.417507 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.417585 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11636,"thread_start_us":61,"threads_started":1}
I20260926 22:46:52.420343 27798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:46:52.422534 27798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:46:52.430573 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.425419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45399 (local address 127.0.0.1:32854)
0926 22:46:52.426247 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:52.426284 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:52.426347 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:52.426636 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:52.426639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:52.426641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:52.426763 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.426766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.428014 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.428015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.429302 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.429305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.430291 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.430292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.430376 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.430392 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 22:46:52.430443 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:46:52.430490 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":74,"threads_started":1}
I20260926 22:46:52.431246 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.425664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.14.184.193:45399)
0926 22:46:52.426468 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.426472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.426482 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:46:52.426514 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.426516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.426531 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.426608 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:52.426837 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.427902 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.429407 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.430160 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.430993 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.431060 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.431085 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.431132 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":70,"threads_started":1}
I20260926 22:46:52.431968 27931 tablet_service.cc:1467] Processing CreateTablet for tablet 5f441ba7e5d742b88cfb13f115e5b0e1 (DEFAULT_TABLE table=test-table [id=26fe0e5b86c546d5a3b21baaa5776fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:52.432314 27931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f441ba7e5d742b88cfb13f115e5b0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:52.434942 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.425656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38429 (local address 127.0.0.1:53380)
0926 22:46:52.430789 (+ 5133us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:52.430797 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:52.430871 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:52.431186 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:52.431187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:52.431188 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:52.431220 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.431222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.432544 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.432545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.433660 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.433663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.434708 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.434710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.434773 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.434777 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:52.434837 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:46:52.434876 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5124,"thread_start_us":66,"threads_started":1}
I20260926 22:46:52.435020 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.425935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.14.184.194:38429)
0926 22:46:52.426147 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.426152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.430845 (+ 4693us) server_negotiation.cc:407] Connection header received
0926 22:46:52.430996 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.431000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.431017 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.431102 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:52.431305 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.432424 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.433766 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.434564 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.434639 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.434728 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.434862 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.434901 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":63,"threads_started":1}
I20260926 22:46:52.435706 28298 tablet_bootstrap.cc:492] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: Bootstrap starting.
I20260926 22:46:52.435997 28065 tablet_service.cc:1467] Processing CreateTablet for tablet 5f441ba7e5d742b88cfb13f115e5b0e1 (DEFAULT_TABLE table=test-table [id=26fe0e5b86c546d5a3b21baaa5776fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:52.436345 28065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f441ba7e5d742b88cfb13f115e5b0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:52.436828 28298 tablet_bootstrap.cc:654] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:52.437119 28298 log.cc:824] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:52.438951 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.425462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:36727 (local address 127.0.0.1:60968)
0926 22:46:52.430734 (+ 5272us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:52.430744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:52.430802 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:52.431072 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:52.431073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:52.431075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:52.431110 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.431111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.436656 (+ 5545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.436657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.437638 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.437641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.438659 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.438660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.438781 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.438785 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:52.438834 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 22:46:52.438874 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5254,"thread_start_us":68,"threads_started":1}
I20260926 22:46:52.439234 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.425863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.14.184.195:36727)
0926 22:46:52.426761 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.426765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.430863 (+ 4098us) server_negotiation.cc:407] Connection header received
0926 22:46:52.430903 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.430906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.430923 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.430994 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:52.431227 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.436567 (+ 5340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.437749 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.438486 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.438564 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.438620 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.439096 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.439145 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":50,"threads_started":1}
I20260926 22:46:52.439925 28199 tablet_service.cc:1467] Processing CreateTablet for tablet 5f441ba7e5d742b88cfb13f115e5b0e1 (DEFAULT_TABLE table=test-table [id=26fe0e5b86c546d5a3b21baaa5776fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:52.440287 28199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f441ba7e5d742b88cfb13f115e5b0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:52.440996 28298 tablet_bootstrap.cc:492] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: No bootstrap required, opened a new log
I20260926 22:46:52.441053 28298 ts_tablet_manager.cc:1397] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 22:46:52.443506 28298 raft_consensus.cc:348] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.443657 28298 raft_consensus.cc:374] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:52.443686 28298 raft_consensus.cc:729] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710f32907ecc4ba3ad7bb783ab920511, State: Initialized, Role: FOLLOWER
I20260926 22:46:52.443929 28298 consensus_queue.cc:260] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.444370 28298 ts_tablet_manager.cc:1428] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:46:52.444489 27997 heartbeater.cc:503] Master 127.14.184.254:33953 was elected leader, sending a full tablet report...
I20260926 22:46:52.446796 28300 tablet_bootstrap.cc:492] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Bootstrap starting.
I20260926 22:46:52.446842 28301 tablet_bootstrap.cc:492] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Bootstrap starting.
I20260926 22:46:52.447324 28300 tablet_bootstrap.cc:654] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:52.447494 28301 tablet_bootstrap.cc:654] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:52.447621 28300 log.cc:824] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:52.447801 28301 log.cc:824] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:52.451046 28300 tablet_bootstrap.cc:492] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: No bootstrap required, opened a new log
I20260926 22:46:52.451150 28300 ts_tablet_manager.cc:1397] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 22:46:52.451311 28301 tablet_bootstrap.cc:492] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: No bootstrap required, opened a new log
I20260926 22:46:52.451373 28301 ts_tablet_manager.cc:1397] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 22:46:52.453617 28300 raft_consensus.cc:348] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.453889 28300 raft_consensus.cc:374] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:52.453984 28300 raft_consensus.cc:729] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a009a398a0b491786fc5aa80ce11308, State: Initialized, Role: FOLLOWER
I20260926 22:46:52.453998 28301 raft_consensus.cc:348] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.454190 28300 consensus_queue.cc:260] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.454239 28301 raft_consensus.cc:374] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:52.454290 28301 raft_consensus.cc:729] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b66247694594d9caed0a66b1b1388ca, State: Initialized, Role: FOLLOWER
I20260926 22:46:52.454437 28301 consensus_queue.cc:260] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.454558 28300 ts_tablet_manager.cc:1428] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:46:52.454762 28131 heartbeater.cc:503] Master 127.14.184.254:33953 was elected leader, sending a full tablet report...
I20260926 22:46:52.454770 28301 ts_tablet_manager.cc:1428] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:46:52.454877 28265 heartbeater.cc:503] Master 127.14.184.254:33953 was elected leader, sending a full tablet report...
W20260926 22:46:52.499572 27998 tablet.cc:2369] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 22:46:52 dist-test-slave-3986 krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for HTTP/127.14.184.212@KRBTEST.COM
W20260926 22:46:52.628561 28266 tablet.cc:2369] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:46:52.673595 28132 tablet.cc:2369] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:46:52.762909 28302 raft_consensus.cc:482] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:46:52.763079 28302 raft_consensus.cc:504] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.763512 28302 leader_election.cc:288] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b66247694594d9caed0a66b1b1388ca (127.14.184.195:36727), 0a009a398a0b491786fc5aa80ce11308 (127.14.184.194:38429)
I20260926 22:46:52.767678 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.763892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:39937 (local address 127.14.184.195:36727)
0926 22:46:52.763949 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.763951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.764052 (+ 101us) server_negotiation.cc:407] Connection header received
0926 22:46:52.764145 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.764146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.764154 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.764182 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:52.764370 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.765104 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.766715 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.767270 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.767319 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.767327 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.767572 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.767609 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:46:52.767701 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.763760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59015 (local address 127.14.184.194:38429)
0926 22:46:52.763810 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.763813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.763902 (+ 89us) server_negotiation.cc:407] Connection header received
0926 22:46:52.764023 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.764024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.764035 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.764068 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:52.764335 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.765135 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.766711 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.767270 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.767319 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.767327 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.767572 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.767611 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:46:52.767959 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.763666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38429 (local address 127.14.184.193:59015)
0926 22:46:52.763835 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:52.763847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:52.763931 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:52.764143 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:52.764144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:52.764145 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:52.764249 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.764251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.765243 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.765244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.766578 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.766581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.767425 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.767426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.767490 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.767493 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:46:52.767806 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 22:46:52.767864 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":50,"thread_start_us":96,"threads_started":1}
I20260926 22:46:52.768255 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.763676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:36727 (local address 127.14.184.193:39937)
0926 22:46:52.763997 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:52.764007 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:52.764071 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:52.764243 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:52.764243 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:52.764244 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:52.764294 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.764295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.765214 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.765215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.766578 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:52.766581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.767408 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.767409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:52.767482 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.767486 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:52.768124 (+ 638us) client_negotiation.cc:240] Negotiation successful
0926 22:46:52.768163 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":92,"thread_start_us":92,"threads_started":1}
I20260926 22:46:52.768608 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f441ba7e5d742b88cfb13f115e5b0e1" candidate_uuid: "710f32907ecc4ba3ad7bb783ab920511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b66247694594d9caed0a66b1b1388ca" is_pre_election: true
I20260926 22:46:52.768776 28219 raft_consensus.cc:2454] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710f32907ecc4ba3ad7bb783ab920511 in term 0.
I20260926 22:46:52.768989 28085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f441ba7e5d742b88cfb13f115e5b0e1" candidate_uuid: "710f32907ecc4ba3ad7bb783ab920511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a009a398a0b491786fc5aa80ce11308" is_pre_election: true
I20260926 22:46:52.769150 28085 raft_consensus.cc:2454] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710f32907ecc4ba3ad7bb783ab920511 in term 0.
I20260926 22:46:52.769371 27886 leader_election.cc:302] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b66247694594d9caed0a66b1b1388ca, 710f32907ecc4ba3ad7bb783ab920511; no voters:
I20260926 22:46:52.769531 28302 raft_consensus.cc:2781] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:46:52.769572 28302 raft_consensus.cc:482] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:46:52.769587 28302 raft_consensus.cc:3037] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:52.770267 28302 raft_consensus.cc:504] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.770429 28302 leader_election.cc:288] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [CANDIDATE]: Term 1 election: Requested vote from peers 0b66247694594d9caed0a66b1b1388ca (127.14.184.195:36727), 0a009a398a0b491786fc5aa80ce11308 (127.14.184.194:38429)
I20260926 22:46:52.770622 28085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f441ba7e5d742b88cfb13f115e5b0e1" candidate_uuid: "710f32907ecc4ba3ad7bb783ab920511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a009a398a0b491786fc5aa80ce11308"
I20260926 22:46:52.770726 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f441ba7e5d742b88cfb13f115e5b0e1" candidate_uuid: "710f32907ecc4ba3ad7bb783ab920511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b66247694594d9caed0a66b1b1388ca"
I20260926 22:46:52.770727 28085 raft_consensus.cc:3037] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:52.770780 28219 raft_consensus.cc:3037] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:52.771556 28085 raft_consensus.cc:2454] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710f32907ecc4ba3ad7bb783ab920511 in term 1.
I20260926 22:46:52.771556 28219 raft_consensus.cc:2454] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710f32907ecc4ba3ad7bb783ab920511 in term 1.
I20260926 22:46:52.771699 27886 leader_election.cc:302] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b66247694594d9caed0a66b1b1388ca, 710f32907ecc4ba3ad7bb783ab920511; no voters:
I20260926 22:46:52.771909 28302 raft_consensus.cc:2781] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:52.772038 28302 raft_consensus.cc:686] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [term 1 LEADER]: Becoming Leader. State: Replica: 710f32907ecc4ba3ad7bb783ab920511, State: Running, Role: LEADER
I20260926 22:46:52.772136 28302 consensus_queue.cc:237] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } }
I20260926 22:46:52.772974 27798 catalog_manager.cc:5495] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 reported cstate change: term changed from 0 to 1, leader changed from <none> to 710f32907ecc4ba3ad7bb783ab920511 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "710f32907ecc4ba3ad7bb783ab920511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710f32907ecc4ba3ad7bb783ab920511" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 } health_report { overall_health: UNKNOWN } } }
I20260926 22:46:52.784320 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.781855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.14.184.193:45399)
0926 22:46:52.781897 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:52.781900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:52.781909 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:46:52.781972 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:52.781973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:52.781989 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:52.782074 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:46:52.782231 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.782767 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.783538 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:52.783756 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:52.783831 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:52.784002 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:46:52.784103 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:46:52.784197 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:52.784221 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 22:46:52.784242 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:46:52.786160 28085 raft_consensus.cc:1261] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 1 FOLLOWER]: Refusing update from remote peer 710f32907ecc4ba3ad7bb783ab920511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:52.786159 28219 raft_consensus.cc:1261] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 1 FOLLOWER]: Refusing update from remote peer 710f32907ecc4ba3ad7bb783ab920511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:52.786510 28318 consensus_queue.cc:1035] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b66247694594d9caed0a66b1b1388ca" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:52.786659 28302 consensus_queue.cc:1035] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 710f32907ecc4ba3ad7bb783ab920511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a009a398a0b491786fc5aa80ce11308" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:52.792876 28323 mvcc.cc:205] Tried to move back new op lower bound from 7333735681169506304 to 7333735681115234304. Current Snapshot: MvccSnapshot[applied={T|T < 7333735681169506304}]
I20260926 22:46:52.793931 27799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:46:52.794039 27799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:46:52.794770 27799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d8709dc1ce5485cb74d31055bfcf4a2: Sending DeleteTablet for 3 replicas of tablet 5f441ba7e5d742b88cfb13f115e5b0e1
I20260926 22:46:52.795120 27931 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f441ba7e5d742b88cfb13f115e5b0e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:52 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:32854
I20260926 22:46:52.795127 28065 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f441ba7e5d742b88cfb13f115e5b0e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:52 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:53380
I20260926 22:46:52.795475 28199 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f441ba7e5d742b88cfb13f115e5b0e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:52 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:60968
I20260926 22:46:52.795768 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 27837
I20260926 22:46:52.796653 28332 tablet_replica.cc:330] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: stopping tablet replica
I20260926 22:46:52.796759 28332 raft_consensus.cc:2229] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:46:52.796859 28332 raft_consensus.cc:2258] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:46:52.797025 28333 tablet_replica.cc:330] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: stopping tablet replica
I20260926 22:46:52.797120 28333 raft_consensus.cc:2229] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:46:52.797223 28333 raft_consensus.cc:2258] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:46:52.797616 28332 ts_tablet_manager.cc:1905] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:46:52.799360 28332 ts_tablet_manager.cc:1918] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:46:52.799470 28332 log.cc:1196] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/5f441ba7e5d742b88cfb13f115e5b0e1
I20260926 22:46:52.799561 28333 ts_tablet_manager.cc:1905] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:46:52.799772 28332 ts_tablet_manager.cc:1939] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0a009a398a0b491786fc5aa80ce11308: Deleting consensus metadata
I20260926 22:46:52.800182 27784 catalog_manager.cc:4841] TS 0a009a398a0b491786fc5aa80ce11308 (127.14.184.194:38429): tablet 5f441ba7e5d742b88cfb13f115e5b0e1 (table test-table [id=26fe0e5b86c546d5a3b21baaa5776fa5]) successfully deleted
I20260926 22:46:52.801139 28333 ts_tablet_manager.cc:1918] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:46:52.801222 28333 log.cc:1196] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/5f441ba7e5d742b88cfb13f115e5b0e1
I20260926 22:46:52.801520 28333 ts_tablet_manager.cc:1939] T 5f441ba7e5d742b88cfb13f115e5b0e1 P 0b66247694594d9caed0a66b1b1388ca: Deleting consensus metadata
I20260926 22:46:52.801920 27785 catalog_manager.cc:4841] TS 0b66247694594d9caed0a66b1b1388ca (127.14.184.195:36727): tablet 5f441ba7e5d742b88cfb13f115e5b0e1 (table test-table [id=26fe0e5b86c546d5a3b21baaa5776fa5]) successfully deleted
W20260926 22:46:52.805665 27783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:45399 (error 108)
I20260926 22:46:52.806295 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28001
I20260926 22:46:52.806298 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:52.806147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45399 (local address 127.0.0.1:32862)
0926 22:46:52.806217 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:52.806261 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:45399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 22:46:52.808821 27783 catalog_manager.cc:4584] TS 710f32907ecc4ba3ad7bb783ab920511 (127.14.184.193:45399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f441ba7e5d742b88cfb13f115e5b0e1: Network error: Client connection negotiation failed: client connection to 127.14.184.193:45399: connect: Connection refused (error 111)
I20260926 22:46:52.811964 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28134
I20260926 22:46:52.817587 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 27766
I20260926 22:46:52.823439 15075 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 22:46:53.158793 15075 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 22:46:53.158905 15075 mini_ranger.cc:66] Stopping Ranger...
I20260926 22:46:53.502964 15075 mini_ranger.cc:68] Stopped Ranger
2026-09-26 22:46:53.503 UTC [26833] LOG: received smart shutdown request
2026-09-26 22:46:53.506 UTC [26833] LOG: background worker "logical replication launcher" (PID 26842) exited with exit code 1
2026-09-26 22:46:53.507 UTC [26837] LOG: shutting down
2026-09-26 22:46:53.663 UTC [26833] LOG: database system is shut down
2026-09-26T22:46:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47157 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:46:53 dist-test-slave-3986 krb5kdc[28343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:46:53 dist-test-slave-3986 krb5kdc[28343](info): set up 2 sockets
Sep 26 22:46:53 dist-test-slave-3986 krb5kdc[28343](info): commencing operation
krb5kdc: starting...
W20260926 22:46:54.790961 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:46:54 dist-test-slave-3986 krb5kdc[28343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:46:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:46:54.905336 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44863
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:44863,127.14.184.253:39357,127.14.184.252:46863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:54.983969 28356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:54.984220 28356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:54.984261 28356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:54.985396 28356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:54.985467 28356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:54.985488 28356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:54.985508 28356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:54.985538 28356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:54.987073 28356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:44863,127.14.184.253:39357,127.14.184.252:46863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:54.987339 28356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:54.987558 28356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:54.992415 28361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:54.992420 28364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:54.992448 28356 server_base.cc:1034] running on GCE node
W20260926 22:46:54.992518 28362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:54.992946 28356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:54.993235 28356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:54.994401 28356 hybrid_clock.cc:648] HybridClock initialized: now 1790462814994376 us; error 36 us; skew 500 ppm
Sep 26 22:46:54 dist-test-slave-3986 krb5kdc[28343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:54.999864 28356 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:46:55.000314 28356 webserver.cc:460] Webserver started at http://127.14.184.254:39333/ using document root <none> and password file <none>
I20260926 22:46:55.000489 28356 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:55.000528 28356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:55.000614 28356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:55.001611 28356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "e1de237e3f3940b0a6ed3b425ea42af6"
format_stamp: "Formatted at 2026-09-26 22:46:54 on dist-test-slave-3986"
server_key: "44f52461a945b56ef92ac0453827aca8"
server_key_iv: "a52a0fbf40cf220e07e31338e9702115"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.001904 28356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "e1de237e3f3940b0a6ed3b425ea42af6"
format_stamp: "Formatted at 2026-09-26 22:46:54 on dist-test-slave-3986"
server_key: "44f52461a945b56ef92ac0453827aca8"
server_key_iv: "a52a0fbf40cf220e07e31338e9702115"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.003235 28356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:46:55.003921 28371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:55.004112 28356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:55.004201 28356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "e1de237e3f3940b0a6ed3b425ea42af6"
format_stamp: "Formatted at 2026-09-26 22:46:54 on dist-test-slave-3986"
server_key: "44f52461a945b56ef92ac0453827aca8"
server_key_iv: "a52a0fbf40cf220e07e31338e9702115"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.004263 28356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:55.028065 28356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:55.030380 28356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:55.030496 28356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:55.034440 28356 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:44863
I20260926 22:46:55.034497 28423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:44863 every 8 connection(s)
I20260926 22:46:55.034875 28356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:46:55.035753 28424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:55.037245 28424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.039718 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28356
I20260926 22:46:55.039834 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:46:55.039999 15075 external_mini_cluster.cc:1428] Setting key 6edf0e4b836f9f44d300ea6f120d8682
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:55.047808 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.037902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.14.184.254:44863)
0926 22:46:55.038059 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.038067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.038228 (+ 161us) server_negotiation.cc:407] Connection header received
0926 22:46:55.038733 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.038735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.038751 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.038868 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.039396 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.040329 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.041623 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.042135 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.042263 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.044685 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.044691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.044691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.044697 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.046182 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.046625 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.046628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.046628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.046762 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.047021 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.047021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.047021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.047212 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.047284 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.047516 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.047646 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.14.184.253@KRBTEST.COM; defaulting to no policy
I20260926 22:46:55.047986 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.037888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44863 (local address 127.0.0.1:37672)
0926 22:46:55.038117 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.038144 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.038565 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.038895 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.038898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.039136 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.039304 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.039308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.040423 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.040424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.041538 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.041541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.042235 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.042236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.042296 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.042857 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.042864 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.044599 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.046269 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.046276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.046278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.046538 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.046830 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.046831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.046831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.046947 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.047288 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.047295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.047375 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.047495 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.047740 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":82,"thread_start_us":87,"threads_started":1}
I20260926 22:46:55.048677 28424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.049067 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.048958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37364)
0926 22:46:55.048995 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.049049 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260926 22:46:55.049207 28374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
I20260926 22:46:55.049593 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.049510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37376)
0926 22:46:55.049555 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.049579 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 22:46:55.049767 28424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:39357: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab.
Entry for principal kudu/127.14.184.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab.
I20260926 22:46:55.063476 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:39357
--webserver_interface=127.14.184.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab
--principal=kudu/127.14.184.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:44863,127.14.184.253:39357,127.14.184.252:46863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:46:55.099001 28424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } attempt: 1
I20260926 22:46:55.099506 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.099346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37388)
0926 22:46:55.099418 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.099467 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 22:46:55.099942 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.099828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37398)
0926 22:46:55.099879 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.099922 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 22:46:55.100028 28424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:39357: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
W20260926 22:46:55.147468 28431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:55.147653 28431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:55.147675 28431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:55.148785 28431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:55.148823 28431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:55.148837 28431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:55.148850 28431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:55.148886 28431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:55.150274 28431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:44863,127.14.184.253:39357,127.14.184.252:46863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab
--principal=kudu/127.14.184.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:39357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.184.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:55.150514 28431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:55.150772 28431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:55.155421 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:55.155490 28436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:55.155421 28439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:55.155635 28431 server_base.cc:1034] running on GCE node
I20260926 22:46:55.155862 28431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:55.156131 28431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:55.157276 28431 hybrid_clock.cc:648] HybridClock initialized: now 1790462815157263 us; error 29 us; skew 500 ppm
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:55.159757 28431 init.cc:377] Logged in from keytab as kudu/127.14.184.253@KRBTEST.COM (short username kudu)
I20260926 22:46:55.160187 28431 webserver.cc:460] Webserver started at http://127.14.184.253:39985/ using document root <none> and password file <none>
I20260926 22:46:55.160362 28431 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:55.160401 28431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:55.160485 28431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:55.161509 28431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data/instance:
uuid: "cc9517e0847341698a7c7463b14077ae"
format_stamp: "Formatted at 2026-09-26 22:46:55 on dist-test-slave-3986"
server_key: "72917108927302deffae982c968fa58f"
server_key_iv: "1df874828c1047abc531adaf8b0e8037"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.161816 28431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal/instance:
uuid: "cc9517e0847341698a7c7463b14077ae"
format_stamp: "Formatted at 2026-09-26 22:46:55 on dist-test-slave-3986"
server_key: "72917108927302deffae982c968fa58f"
server_key_iv: "1df874828c1047abc531adaf8b0e8037"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.163110 28431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 22:46:55.163753 28446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:55.163941 28431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:55.164011 28431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal
uuid: "cc9517e0847341698a7c7463b14077ae"
format_stamp: "Formatted at 2026-09-26 22:46:55 on dist-test-slave-3986"
server_key: "72917108927302deffae982c968fa58f"
server_key_iv: "1df874828c1047abc531adaf8b0e8037"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.164098 28431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:55.168154 28424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } attempt: 2
I20260926 22:46:55.168658 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.168418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37406)
0926 22:46:55.168594 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.168629 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260926 22:46:55.169008 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.168891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37414)
0926 22:46:55.168939 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.168977 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260926 22:46:55.169224 28424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:39357: Network error: Client connection negotiation failed: client connection to 127.14.184.253:39357: connect: Connection refused (error 111)
I20260926 22:46:55.192911 28431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:55.195003 28431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:55.195155 28431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:55.199120 28431 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.253:39357
I20260926 22:46:55.199159 28498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.253:39357 every 8 connection(s)
I20260926 22:46:55.199559 28431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
I20260926 22:46:55.200394 28499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:55.201939 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28431
I20260926 22:46:55.202018 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-1/wal/instance
I20260926 22:46:55.201967 28499 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.202198 15075 external_mini_cluster.cc:1428] Setting key 58bb5b22b85928f4d584b206bca58fa5
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.252@KRBTEST.COM; defaulting to no policy
I20260926 22:46:55.211624 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.202737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.14.184.254:44863)
0926 22:46:55.202780 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.202781 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.202984 (+ 203us) server_negotiation.cc:407] Connection header received
0926 22:46:55.203497 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.203499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.203517 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.203587 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.204434 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.204904 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.205883 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.206093 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.206162 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.208685 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.208687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.208687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.208693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.210106 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.210540 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.210541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.210541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.210600 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.211017 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.211019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.211019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.211147 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.211217 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.211484 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.211564 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:46:55.211722 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.202702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44863 (local address 127.0.0.1:37674)
0926 22:46:55.202864 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.202902 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.203360 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.203673 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.203678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.204065 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.204343 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.204349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.204979 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.204980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.205781 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.205783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.206180 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.206181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.206305 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.206845 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.206877 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.208595 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.210181 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.210188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.210190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.210453 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.210789 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.210790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.210791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.210942 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.211217 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.211222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.211330 (+ 108us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.211432 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.211546 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260926 22:46:55.212268 28499 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.252@KRBTEST.COM" created.
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:46:55.220209 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.212535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37416)
0926 22:46:55.212712 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.212722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.212776 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.212975 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.212976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.213077 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.213118 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.213119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.214336 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.214338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.215239 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.215242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.215682 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.215683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.215764 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.216145 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.216151 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.217409 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.219175 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.219175 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.219176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.219344 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.219557 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.219558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.219558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.219610 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.219895 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.219896 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.219934 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.220001 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.220094 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260926 22:46:55.220294 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.212597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.14.184.253:39357)
0926 22:46:55.212718 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.212724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.212772 (+ 48us) server_negotiation.cc:407] Connection header received
0926 22:46:55.212829 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.212832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.212844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.212942 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.213196 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.214238 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.215330 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.215601 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.215652 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.217491 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.217499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.217500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.217506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.219077 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.219434 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.219437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.219437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.219491 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.219681 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.219682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.219683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.219817 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.219905 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.220030 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.220170 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":92,"thread_start_us":69,"threads_started":1}
I20260926 22:46:55.220769 28499 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.221096 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.220968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33254)
0926 22:46:55.221030 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.221073 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:46:55.221246 28447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
I20260926 22:46:55.221601 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.221535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33270)
0926 22:46:55.221561 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.221582 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260926 22:46:55.221701 28499 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:46863: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab.
Entry for principal kudu/127.14.184.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab.
I20260926 22:46:55.227063 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:46863
--webserver_interface=127.14.184.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab
--principal=kudu/127.14.184.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:44863,127.14.184.253:39357,127.14.184.252:46863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:46:55.260382 28424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } attempt: 3
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:46:55.267695 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.260802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.14.184.253:39357)
0926 22:46:55.260869 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.260870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.260881 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:46:55.260934 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.260935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.260946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.261017 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.261345 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.261875 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.262573 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.262854 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.262932 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.264999 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.265001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.265001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.265009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.266452 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.266897 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.266898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.266899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.266957 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.267165 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.267165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.267166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.267274 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.267345 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.267508 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.267583 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 22:46:55.267684 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.260736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37432)
0926 22:46:55.260781 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.260800 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.260870 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.261023 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.261023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.261186 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.261256 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.261257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.261961 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.261962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.262476 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.262478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.262932 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.262933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.263014 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.263474 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.263479 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.264909 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.266557 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.266558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.266558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.266812 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.267025 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.267026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.267026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.267084 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.267350 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.267351 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.267398 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.267483 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.267584 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 22:46:55.268383 28424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.268739 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.268626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33284)
0926 22:46:55.268666 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.268719 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 22:46:55.269029 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.268967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33286)
0926 22:46:55.268996 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.269016 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 22:46:55.269129 28424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:46863: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
I20260926 22:46:55.270918 28499 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } attempt: 1
I20260926 22:46:55.271247 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.271132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33302)
0926 22:46:55.271181 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.271225 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:46:55.271526 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.271440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33306)
0926 22:46:55.271482 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.271506 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:46:55.271616 28499 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:46863: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
I20260926 22:46:55.300269 28424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } attempt: 1
I20260926 22:46:55.300741 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.300593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33312)
0926 22:46:55.300653 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.300712 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 22:46:55.301074 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.300980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33320)
0926 22:46:55.301024 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.301051 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:46:55.301162 28424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:46863: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
W20260926 22:46:55.306351 28506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:55.306524 28506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:55.306546 28506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:55.307773 28506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:55.307813 28506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:55.307828 28506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:55.307876 28506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:55.307907 28506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:55.309309 28506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:44863,127.14.184.253:39357,127.14.184.252:46863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab
--principal=kudu/127.14.184.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:46863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.184.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:55.309540 28506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:55.309756 28506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:55.314400 28512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:55.314432 28506 server_base.cc:1034] running on GCE node
W20260926 22:46:55.314404 28514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:55.314407 28511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:55.314893 28506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:55.315164 28506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:55.316327 28506 hybrid_clock.cc:648] HybridClock initialized: now 1790462815316318 us; error 28 us; skew 500 ppm
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:55.318955 28506 init.cc:377] Logged in from keytab as kudu/127.14.184.252@KRBTEST.COM (short username kudu)
I20260926 22:46:55.319386 28506 webserver.cc:460] Webserver started at http://127.14.184.252:38821/ using document root <none> and password file <none>
I20260926 22:46:55.319592 28506 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:55.319634 28506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:55.319779 28506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:55.320787 28506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data/instance:
uuid: "1d320e18669343b183f099d5d75816d8"
format_stamp: "Formatted at 2026-09-26 22:46:55 on dist-test-slave-3986"
server_key: "d101fd88520d3e9f8e3725da68d0a94c"
server_key_iv: "34bca1f8335c3e53dc6fbfadd2a51dd0"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.321130 28506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal/instance:
uuid: "1d320e18669343b183f099d5d75816d8"
format_stamp: "Formatted at 2026-09-26 22:46:55 on dist-test-slave-3986"
server_key: "d101fd88520d3e9f8e3725da68d0a94c"
server_key_iv: "34bca1f8335c3e53dc6fbfadd2a51dd0"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.322347 28506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:46:55.323061 28521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:55.323240 28506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:55.323323 28506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal
uuid: "1d320e18669343b183f099d5d75816d8"
format_stamp: "Formatted at 2026-09-26 22:46:55 on dist-test-slave-3986"
server_key: "d101fd88520d3e9f8e3725da68d0a94c"
server_key_iv: "34bca1f8335c3e53dc6fbfadd2a51dd0"
server_key_version: "encryptionkey@0"
I20260926 22:46:55.323388 28506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:55.339752 28499 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } attempt: 2
I20260926 22:46:55.340165 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.340040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33336)
0926 22:46:55.340098 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.340136 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 22:46:55.340497 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.340418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33348)
0926 22:46:55.340457 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.340481 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 22:46:55.340615 28499 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:46863: Network error: Client connection negotiation failed: client connection to 127.14.184.252:46863: connect: Connection refused (error 111)
I20260926 22:46:55.348979 28506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:55.352334 28506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:55.352483 28506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:55.356361 28506 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.252:46863
I20260926 22:46:55.356395 28573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.252:46863 every 8 connection(s)
I20260926 22:46:55.356724 28506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
I20260926 22:46:55.357538 28574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:55.359138 28574 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.361245 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28506
I20260926 22:46:55.361356 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790462668484683-15075-0/minicluster-data/master-2/wal/instance
I20260926 22:46:55.361552 15075 external_mini_cluster.cc:1428] Setting key fb2bd7a2782714b5a41d0ff042fa8366
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:46:55.368299 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.359797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.14.184.254:44863)
0926 22:46:55.359854 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.359856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.360022 (+ 166us) server_negotiation.cc:407] Connection header received
0926 22:46:55.360464 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.360465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.360476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.360541 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.361113 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.361695 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.362953 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.363163 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.363212 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.365350 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.365351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.365352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.365359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.366848 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.367307 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.367308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.367308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.367370 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.367621 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.367621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.367622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.367745 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.367795 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.368145 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.368235 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:46:55.368510 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.359745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:44863 (local address 127.0.0.1:37678)
0926 22:46:55.359901 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.359941 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.360359 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.360611 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.360614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.360837 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.361018 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.361021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.361791 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.361791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.362856 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.362858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.363239 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.363240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.363376 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.363809 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.363819 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.365232 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.366940 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.366945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.366947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.367148 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.367442 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.367442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.367442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.367532 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.367808 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.367813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.367992 (+ 179us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.368108 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.368275 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260926 22:46:55.369027 28574 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.370625 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.361925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33358 (local address 127.14.184.252:46863)
0926 22:46:55.362089 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.362093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.362110 (+ 17us) server_negotiation.cc:407] Connection header received
0926 22:46:55.362159 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.362161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.362180 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.362260 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.362856 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.363802 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.364590 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.364867 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.364954 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.367671 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.367678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.367678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.367691 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.369481 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.369788 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.369791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.369791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.369846 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.370067 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.370068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.370068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.370202 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.370355 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.370444 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.370518 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:55.375742 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.369330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37436 (local address 127.14.184.253:39357)
0926 22:46:55.369367 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.369369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.369377 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:46:55.369426 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.369427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.369435 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.369471 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.369729 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.370171 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.370939 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.371191 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.371282 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.373018 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.373019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.373020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.373026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.374632 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.374994 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.374995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.374996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.375057 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.375309 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.375309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.375309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.375414 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.375465 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.375627 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.375682 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:46:55.375839 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.369273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:39357 (local address 127.0.0.1:37436)
0926 22:46:55.369321 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.369329 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.369364 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.369477 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.369478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.369607 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.369641 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.369643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.370250 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.370251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.370861 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.370864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.371281 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.371282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.371343 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.371734 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.371739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.372936 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.374765 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.374765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.374766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.374918 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.375148 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.375150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.375150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.375230 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.375482 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.375483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.375532 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.375613 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.375729 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 22:46:55.376276 28574 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:46:55.376279 28424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } attempt: 2
I20260926 22:46:55.378904 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.371368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.14.184.254:44863)
0926 22:46:55.371414 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.371416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.371470 (+ 54us) server_negotiation.cc:407] Connection header received
0926 22:46:55.371527 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.371528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.371536 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.371616 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.372170 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.372581 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.373391 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.373624 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.373676 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.376258 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.376259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.376259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.376266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.377821 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.378194 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.378195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.378195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.378244 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.378445 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.378445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.378446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.378528 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.378654 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.378778 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.378847 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:46:55.385141 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.376504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33376)
0926 22:46:55.376734 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.376744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.376787 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.376918 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.376919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.377101 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.377130 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.377132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.379005 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.379006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.379646 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.379648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.380024 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.380025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.380071 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.380537 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.380541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.381981 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.384027 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.384028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.384028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.384247 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.384460 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.384460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.384461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.384513 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.384845 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.384846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.384883 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.384944 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.385028 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260926 22:46:55.385684 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.376590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.14.184.252:46863)
0926 22:46:55.376745 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.376746 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.376776 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:46:55.376836 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.376836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.376848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.376919 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.377231 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.378041 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.379724 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.379946 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.380015 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.382065 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.382067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.382067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.382074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.383946 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.384334 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.384335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.384336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.384388 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.384590 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.384591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.384592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.384757 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.384824 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.385306 (+ 482us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.385383 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260926 22:46:55.386281 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.376511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33370)
0926 22:46:55.376708 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.376721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.376769 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.376970 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.376972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.377120 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.377150 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.377151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.378119 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.378120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.379382 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.379384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.379929 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.379931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.379992 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.380357 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.380362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.382051 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.384736 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.384737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.384737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.384956 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.385289 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.385289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.385290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.385380 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.385969 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.385970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.386020 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.386076 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.386180 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260926 22:46:55.386313 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.376568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.14.184.252:46863)
0926 22:46:55.376605 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.376607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.376856 (+ 249us) server_negotiation.cc:407] Connection header received
0926 22:46:55.376874 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.376874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.376883 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.376958 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.377273 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.378042 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.379495 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.379837 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.380135 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.382123 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.382124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.382124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.382129 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.384654 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.385033 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.385034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.385035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.385095 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.385447 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.385447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.385448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.385572 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.385635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.386125 (+ 490us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.386202 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":19}
I20260926 22:46:55.387306 28574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: Bootstrap starting.
I20260926 22:46:55.387914 28574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:55.387995 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.379477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37448 (local address 127.14.184.253:39357)
0926 22:46:55.379514 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.379515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.379583 (+ 68us) server_negotiation.cc:407] Connection header received
0926 22:46:55.379662 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.379663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.379672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.379699 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.380781 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.381274 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.381988 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.382286 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.382348 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.385153 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.385155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.385155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.385162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.386919 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.387246 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.387247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.387247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.387299 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.387483 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.387485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.387485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.387592 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.387772 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.387809 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.387881 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:46:55.388545 28574 log.cc:824] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:55.389436 28574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: No bootstrap required, opened a new log
I20260926 22:46:55.389647 28424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: Bootstrap starting.
I20260926 22:46:55.389876 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:46:55.390275 28424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:55.390563 28424 log.cc:824] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:55.391422 28424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: No bootstrap required, opened a new log
I20260926 22:46:55.391585 28574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.391768 28574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:55.391815 28574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d320e18669343b183f099d5d75816d8, State: Initialized, Role: FOLLOWER
I20260926 22:46:55.391952 28574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.392283 28574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:55.392637 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.392743 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:46:55.393294 28424 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.393433 28424 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:55.393479 28424 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1de237e3f3940b0a6ed3b425ea42af6, State: Initialized, Role: FOLLOWER
I20260926 22:46:55.393599 28424 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.393887 28424 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:55.393864 28600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.393950 28600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 22:46:55.395256 28599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:46:55.395305 28599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 22:46:55.396178 28612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:46:55.396231 28612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:46:55.395064 28506 master_runner.cc:186] Error getting master registration for 127.14.184.254:44863: OK, instance_id { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" instance_seqno: 1790462815033304 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 22:46:55.398007 28506 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 22:46:55.396574 28356 master_runner.cc:186] Error getting master registration for 127.14.184.253:39357: OK, instance_id { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" instance_seqno: 1790462815197951 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 22:46:55.399426 28356 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 22:46:55.401304 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.391693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.14.184.254:44863)
0926 22:46:55.391759 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.391761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.391769 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:46:55.391787 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.391788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.391798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.391849 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.392563 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.393246 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.394176 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.394481 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.394568 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.398081 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.398083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.398083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.398090 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.400052 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.400443 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.400444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.400444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.400502 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.400715 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.400716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.400716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.400840 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.400944 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.401126 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.401213 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:46:55.404174 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.394744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.14.184.252:46863)
0926 22:46:55.394791 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.394793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.396498 (+ 1705us) server_negotiation.cc:407] Connection header received
0926 22:46:55.396605 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.396606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.396617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.396679 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.397253 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.397943 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.399027 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.399284 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.399357 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.400847 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.400848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.400849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.400855 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.403059 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.403420 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.403421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.403421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.403480 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.403674 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.403674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.403675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.403791 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.403899 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.404012 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.404087 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:46:55.407790 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.395571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.14.184.253:39357)
0926 22:46:55.395611 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.395613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.401381 (+ 5768us) server_negotiation.cc:407] Connection header received
0926 22:46:55.401404 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.401405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.401415 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.401462 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.401946 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.402443 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.403372 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.403666 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.403725 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.405104 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.405105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.405105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.405113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.406565 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.406945 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.406946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.406947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.407004 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.407208 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.407209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.407210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.407342 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.407444 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.407594 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.407667 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:46:55.431816 28499 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } attempt: 3
Sep 26 22:46:55 dist-test-slave-3986 krb5kdc[28343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:46:55.439827 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.432103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:46863 (local address 127.0.0.1:33392)
0926 22:46:55.432154 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:55.432164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:55.432215 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:55.432364 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:55.432365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:55.432537 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:55.432585 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.432587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.433488 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.433489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.434310 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:55.434312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.434894 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.434894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:55.434963 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.435430 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:55.435435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:55.436930 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:55.438724 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.438725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.438726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.438939 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.439149 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:55.439150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:55.439150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:55.439250 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:55.439517 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:55.439518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:55.439563 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 22:46:55.439635 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 22:46:55.439725 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 22:46:55.439832 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:55.432134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.14.184.252:46863)
0926 22:46:55.432182 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:55.432184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:55.432195 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:46:55.432268 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:55.432268 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:55.432278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:55.432342 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:55.432666 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.433402 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.434400 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:55.434807 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:55.434871 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:55.437021 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:55.437023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:55.437023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:55.437029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:55.438622 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.439017 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.439018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.439018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.439077 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:55.439319 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:55.439319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:55.439320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:55.439446 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:55.439513 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:55.439674 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 22:46:55.439746 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:46:55.441789 28499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: Bootstrap starting.
I20260926 22:46:55.442523 28499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:55.442868 28499 log.cc:824] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:55.443742 28499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: No bootstrap required, opened a new log
I20260926 22:46:55.445681 28499 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.445832 28499 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:55.445871 28499 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9517e0847341698a7c7463b14077ae, State: Initialized, Role: FOLLOWER
I20260926 22:46:55.446012 28499 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.446301 28499 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:55.446270 28616 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.446444 28616 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: This master's current role is: FOLLOWER
W20260926 22:46:55.448065 28627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:46:55.448163 28627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:46:55.450083 28431 master_runner.cc:263] No leader master found from master cc9517e0847341698a7c7463b14077ae
I20260926 22:46:55.450127 28431 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 22:46:55.531286 28377 tablet.cc:2369] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:46:55.571734 28600 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:46:55.571900 28600 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.572162 28600 leader_election.cc:288] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc9517e0847341698a7c7463b14077ae (127.14.184.253:39357), 1d320e18669343b183f099d5d75816d8 (127.14.184.252:46863)
I20260926 22:46:55.572446 28548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d320e18669343b183f099d5d75816d8" is_pre_election: true
I20260926 22:46:55.572486 28473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9517e0847341698a7c7463b14077ae" is_pre_election: true
I20260926 22:46:55.572587 28548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1de237e3f3940b0a6ed3b425ea42af6 in term 0.
I20260926 22:46:55.572671 28473 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1de237e3f3940b0a6ed3b425ea42af6 in term 0.
I20260926 22:46:55.572764 28372 leader_election.cc:302] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d320e18669343b183f099d5d75816d8, e1de237e3f3940b0a6ed3b425ea42af6; no voters:
I20260926 22:46:55.572904 28600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:46:55.572952 28600 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:46:55.572973 28600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:55.573848 28600 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.573980 28600 leader_election.cc:288] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [CANDIDATE]: Term 1 election: Requested vote from peers cc9517e0847341698a7c7463b14077ae (127.14.184.253:39357), 1d320e18669343b183f099d5d75816d8 (127.14.184.252:46863)
I20260926 22:46:55.574235 28473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9517e0847341698a7c7463b14077ae"
I20260926 22:46:55.574314 28473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:55.574373 28548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d320e18669343b183f099d5d75816d8"
I20260926 22:46:55.574432 28548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:55.575178 28548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1de237e3f3940b0a6ed3b425ea42af6 in term 1.
I20260926 22:46:55.575176 28473 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1de237e3f3940b0a6ed3b425ea42af6 in term 1.
I20260926 22:46:55.575320 28372 leader_election.cc:302] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d320e18669343b183f099d5d75816d8, e1de237e3f3940b0a6ed3b425ea42af6; no voters:
I20260926 22:46:55.575387 28600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:55.575507 28600 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [term 1 LEADER]: Becoming Leader. State: Replica: e1de237e3f3940b0a6ed3b425ea42af6, State: Running, Role: LEADER
I20260926 22:46:55.575584 28600 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } }
I20260926 22:46:55.575927 28600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1de237e3f3940b0a6ed3b425ea42af6. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.575994 28600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:55.576280 28631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:46:55.576444 28631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:46:55.578148 28548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [term 1 FOLLOWER]: Refusing update from remote peer e1de237e3f3940b0a6ed3b425ea42af6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:55.578186 28473 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [term 1 FOLLOWER]: Refusing update from remote peer e1de237e3f3940b0a6ed3b425ea42af6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:55.578529 28600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:55.578689 28600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:55.579864 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1de237e3f3940b0a6ed3b425ea42af6. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.579969 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:46:55.580030 28616 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1de237e3f3940b0a6ed3b425ea42af6. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.580112 28616 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:46:55.580521 28600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.580595 28600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:55.580856 28616 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.580935 28616 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:46:55.581059 28632 mvcc.cc:205] Tried to move back new op lower bound from 7333735692605681664 to 7333735692597673984. Current Snapshot: MvccSnapshot[applied={T|T < 7333735692605681664}]
I20260926 22:46:55.581231 28600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.581303 28600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:55.581488 28634 mvcc.cc:205] Tried to move back new op lower bound from 7333735692605681664 to 7333735692597673984. Current Snapshot: MvccSnapshot[applied={T|T < 7333735692605681664}]
I20260926 22:46:55.581666 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1de237e3f3940b0a6ed3b425ea42af6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 44863 } } peers { permanent_uuid: "cc9517e0847341698a7c7463b14077ae" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 39357 } } peers { permanent_uuid: "1d320e18669343b183f099d5d75816d8" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 46863 } } }
I20260926 22:46:55.581734 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:46:55.581856 28631 catalog_manager.cc:1347] Generated new cluster ID: c96a29cb14b54f1ab24511375e020090
I20260926 22:46:55.581925 28631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:46:55.582438 28633 mvcc.cc:205] Tried to move back new op lower bound from 7333735692605681664 to 7333735692597673984. Current Snapshot: MvccSnapshot[applied={T|T < 7333735692605681664}]
I20260926 22:46:55.586513 28631 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:46:55.587239 28631 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:46:55.598798 28631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1de237e3f3940b0a6ed3b425ea42af6: Generated new TSK 0
I20260926 22:46:55.599118 28631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:46:56.395742 28599 catalog_manager.cc:1259] Loaded cluster ID: c96a29cb14b54f1ab24511375e020090
I20260926 22:46:56.395833 28599 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: loading cluster ID for follower catalog manager: success
I20260926 22:46:56.396683 28599 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: acquiring CA information for follower catalog manager: success
I20260926 22:46:56.396986 28599 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d320e18669343b183f099d5d75816d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:46:56.448627 28627 catalog_manager.cc:1259] Loaded cluster ID: c96a29cb14b54f1ab24511375e020090
I20260926 22:46:56.448726 28627 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: loading cluster ID for follower catalog manager: success
I20260926 22:46:56.449568 28627 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: acquiring CA information for follower catalog manager: success
I20260926 22:46:56.449898 28627 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc9517e0847341698a7c7463b14077ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:46:56.661535 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28356
I20260926 22:46:56.666087 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28431
I20260926 22:46:56.670778 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28506
2026-09-26T22:46:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2903 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:46:56 dist-test-slave-3986 krb5kdc[28648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:46:56 dist-test-slave-3986 krb5kdc[28648](info): set up 2 sockets
Sep 26 22:46:56 dist-test-slave-3986 krb5kdc[28648](info): commencing operation
krb5kdc: starting...
W20260926 22:46:57.695037 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:46:57 dist-test-slave-3986 krb5kdc[28648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:46:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:46:57.806133 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:36293
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:57.884866 28661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:57.885097 28661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:57.885133 28661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:57.886227 28661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:57.886294 28661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:57.886317 28661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:57.886337 28661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:57.886364 28661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:57.887818 28661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:36293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:57.888088 28661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:57.888302 28661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:57.893090 28669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:57.893078 28667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:57.893078 28666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:57.893368 28661 server_base.cc:1034] running on GCE node
I20260926 22:46:57.893605 28661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:57.893891 28661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:57.895056 28661 hybrid_clock.cc:648] HybridClock initialized: now 1790462817895034 us; error 30 us; skew 500 ppm
Sep 26 22:46:57 dist-test-slave-3986 krb5kdc[28648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462817, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:57.897406 28661 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:46:57.897776 28661 webserver.cc:460] Webserver started at http://127.14.184.254:34951/ using document root <none> and password file <none>
I20260926 22:46:57.897964 28661 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:57.898003 28661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:57.898088 28661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:57.899151 28661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "5bbead956a0d4819a2f6972058a9f004"
format_stamp: "Formatted at 2026-09-26 22:46:57 on dist-test-slave-3986"
server_key: "c4c5a7a6a9bb80057870e9c1f1ffc342"
server_key_iv: "0f0198dd3acceba3074a7d73443e29df"
server_key_version: "encryptionkey@0"
I20260926 22:46:57.899444 28661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "5bbead956a0d4819a2f6972058a9f004"
format_stamp: "Formatted at 2026-09-26 22:46:57 on dist-test-slave-3986"
server_key: "c4c5a7a6a9bb80057870e9c1f1ffc342"
server_key_iv: "0f0198dd3acceba3074a7d73443e29df"
server_key_version: "encryptionkey@0"
I20260926 22:46:57.900763 28661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:46:57.901398 28676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:57.901554 28661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:46:57.901643 28661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "5bbead956a0d4819a2f6972058a9f004"
format_stamp: "Formatted at 2026-09-26 22:46:57 on dist-test-slave-3986"
server_key: "c4c5a7a6a9bb80057870e9c1f1ffc342"
server_key_iv: "0f0198dd3acceba3074a7d73443e29df"
server_key_version: "encryptionkey@0"
I20260926 22:46:57.901714 28661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:57.913097 28661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:57.915230 28661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:57.915377 28661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:57.919015 28661 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:36293
I20260926 22:46:57.919066 28728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:36293 every 8 connection(s)
I20260926 22:46:57.919505 28661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:46:57.920127 28729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:57.920491 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28661
I20260926 22:46:57.920578 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:46:57.920783 15075 external_mini_cluster.cc:1428] Setting key eeef8d8c8391aa2f525ac3ebdbd5e968
I20260926 22:46:57.923245 28729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004: Bootstrap starting.
I20260926 22:46:57.923913 28729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:57.924201 28729 log.cc:824] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:57.924899 28729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004: No bootstrap required, opened a new log
I20260926 22:46:57.927161 28729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbead956a0d4819a2f6972058a9f004" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36293 } }
I20260926 22:46:57.927343 28729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:57.927378 28729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbead956a0d4819a2f6972058a9f004, State: Initialized, Role: FOLLOWER
I20260926 22:46:57.927495 28729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbead956a0d4819a2f6972058a9f004" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36293 } }
I20260926 22:46:57.927580 28729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:46:57.927613 28729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:46:57.927659 28729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 22:46:57 dist-test-slave-3986 krb5kdc[28648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:57.928365 28729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbead956a0d4819a2f6972058a9f004" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36293 } }
I20260926 22:46:57.928490 28729 leader_election.cc:302] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bbead956a0d4819a2f6972058a9f004; no voters:
I20260926 22:46:57.928663 28729 leader_election.cc:288] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:46:57.928701 28734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:57.928865 28734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [term 1 LEADER]: Becoming Leader. State: Replica: 5bbead956a0d4819a2f6972058a9f004, State: Running, Role: LEADER
I20260926 22:46:57.928916 28729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:57.929041 28734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbead956a0d4819a2f6972058a9f004" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36293 } }
I20260926 22:46:57.929391 28735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bbead956a0d4819a2f6972058a9f004. Latest consensus state: current_term: 1 leader_uuid: "5bbead956a0d4819a2f6972058a9f004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbead956a0d4819a2f6972058a9f004" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36293 } } }
I20260926 22:46:57.929512 28735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:57.929554 28736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bbead956a0d4819a2f6972058a9f004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbead956a0d4819a2f6972058a9f004" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36293 } } }
I20260926 22:46:57.929675 28736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:57.929819 28742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:46:57.930281 28742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:46:57.931871 28742 catalog_manager.cc:1347] Generated new cluster ID: fc9b27b3e3484acfa2ec7f8a5034ea85
I20260926 22:46:57.931933 28742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:46:57.932288 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:57.921292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.14.184.254:36293)
0926 22:46:57.921544 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:57.921554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:57.921585 (+ 31us) server_negotiation.cc:407] Connection header received
0926 22:46:57.922135 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:57.922143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:57.922201 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:57.922345 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:57.923011 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:57.924351 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:57.925193 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:57.925799 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:57.925919 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:57.928343 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:57.928346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:57.928346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:57.928355 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:57.930605 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:57.931102 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:57.931105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:57.931105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:57.931209 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:57.931455 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:57.931456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:57.931456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:57.931718 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:57.931863 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:57.931986 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 22:46:57.932116 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":62,"threads_started":1}
I20260926 22:46:57.933156 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:46:57.951215 28742 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:46:57.951964 28742 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:46:57.957983 28742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bbead956a0d4819a2f6972058a9f004: Generated new TSK 0
I20260926 22:46:57.958184 28742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:46:57.987072 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28661
2026-09-26T22:46:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1315 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:46:58 dist-test-slave-3986 krb5kdc[28756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:46:58 dist-test-slave-3986 krb5kdc[28756](info): set up 2 sockets
Sep 26 22:46:58 dist-test-slave-3986 krb5kdc[28756](info): commencing operation
krb5kdc: starting...
W20260926 22:46:59.051062 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:46:59.165318 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38729
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:38729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:59.245973 28769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:59.246173 28769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:59.246197 28769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:59.247408 28769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:46:59.247453 28769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:59.247467 28769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:59.247480 28769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:46:59.247493 28769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:46:59.248941 28769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:38729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:59.249183 28769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:59.249414 28769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:59.254321 28777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.254340 28775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.254336 28774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:59.254376 28769 server_base.cc:1034] running on GCE node
I20260926 22:46:59.254853 28769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:59.255151 28769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:59.256320 28769 hybrid_clock.cc:648] HybridClock initialized: now 1790462819256309 us; error 47 us; skew 500 ppm
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:59.258895 28769 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:46:59.259327 28769 webserver.cc:460] Webserver started at http://127.14.184.254:41901/ using document root <none> and password file <none>
I20260926 22:46:59.259547 28769 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:59.259590 28769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:59.259729 28769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:59.260754 28769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "1083b9965d704a788eb40a7b8551d1f6"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "f036eae4d804674321aa9caff32b03c4"
server_key_iv: "dddeba0051ea0ca37211f2ec3a356fe7"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.261106 28769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "1083b9965d704a788eb40a7b8551d1f6"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "f036eae4d804674321aa9caff32b03c4"
server_key_iv: "dddeba0051ea0ca37211f2ec3a356fe7"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.262368 28769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 22:46:59.263206 28784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.263396 28769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 22:46:59.263484 28769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "1083b9965d704a788eb40a7b8551d1f6"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "f036eae4d804674321aa9caff32b03c4"
server_key_iv: "dddeba0051ea0ca37211f2ec3a356fe7"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.263556 28769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:59.283980 28769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:59.286283 28769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:59.286433 28769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:59.290290 28769 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:38729
I20260926 22:46:59.290409 28836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:38729 every 8 connection(s)
I20260926 22:46:59.290719 28769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:46:59.291658 28837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:59.293973 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap starting.
I20260926 22:46:59.294553 28837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:59.294888 28837 log.cc:824] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:59.295562 28837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: No bootstrap required, opened a new log
I20260926 22:46:59.297155 28837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:46:59.297297 28837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:59.297343 28837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1083b9965d704a788eb40a7b8551d1f6, State: Initialized, Role: FOLLOWER
I20260926 22:46:59.297500 28837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:46:59.297562 28837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:46:59.297583 28837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:46:59.297652 28837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:59.298450 28837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:46:59.298563 28837 leader_election.cc:302] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1083b9965d704a788eb40a7b8551d1f6; no voters:
I20260926 22:46:59.298782 28837 leader_election.cc:288] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:46:59.298806 28840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:59.298939 28840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 1083b9965d704a788eb40a7b8551d1f6, State: Running, Role: LEADER
I20260926 22:46:59.299041 28837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:46:59.299069 28840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:46:59.299448 28841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1083b9965d704a788eb40a7b8551d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } } }
I20260926 22:46:59.299569 28841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:59.299451 28842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1083b9965d704a788eb40a7b8551d1f6. Latest consensus state: current_term: 1 leader_uuid: "1083b9965d704a788eb40a7b8551d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } } }
I20260926 22:46:59.299798 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28769
I20260926 22:46:59.299819 28842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:46:59.299908 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:46:59.300120 15075 external_mini_cluster.cc:1428] Setting key da1cc0cef22e4d690b80b685d90129ee
I20260926 22:46:59.300190 28848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:46:59.300684 28848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:46:59.302203 28848 catalog_manager.cc:1347] Generated new cluster ID: ebbfa6f1dc234c40ba369888b22cbaca
I20260926 22:46:59.302242 28848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:59.313375 28848 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:46:59.313990 28848 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:46:59.314186 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.300691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.14.184.254:38729)
0926 22:46:59.302972 (+ 2281us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.302981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.303009 (+ 28us) server_negotiation.cc:407] Connection header received
0926 22:46:59.303137 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.303143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.303199 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.303318 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:59.304098 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.305470 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.306528 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.307614 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.307710 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.310406 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:59.310410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:59.310410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:59.310421 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:59.312434 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.312833 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.312837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.312837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.312945 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.313156 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.313157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.313157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.313426 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:59.313557 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.313790 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.313962 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":2236,"thread_start_us":74,"threads_started":1}
I20260926 22:46:59.320343 28848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Generated new TSK 0
I20260926 22:46:59.320495 28848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:46:59.349212 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38729
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:59.430173 28863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:59.430388 28863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:59.430410 28863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:59.431667 28863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:59.431711 28863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:59.431766 28863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:46:59.433231 28863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:59.433478 28863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:59.433704 28863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:59.438560 28868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.438572 28871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.438577 28869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:59.439034 28863 server_base.cc:1034] running on GCE node
I20260926 22:46:59.439189 28863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:59.439368 28863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:59.440501 28863 hybrid_clock.cc:648] HybridClock initialized: now 1790462819440493 us; error 25 us; skew 500 ppm
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:59.442891 28863 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:46:59.443305 28863 webserver.cc:460] Webserver started at http://127.14.184.193:40115/ using document root <none> and password file <none>
I20260926 22:46:59.443548 28863 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:59.443632 28863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:59.443789 28863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:59.444794 28863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "5552630dca0648349d5f7069f706bd58"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c5a19b0d07c97f080405374fc13e7a34"
server_key_iv: "d79526fdb9bd0e97f544efeb238f9925"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.445161 28863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "5552630dca0648349d5f7069f706bd58"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c5a19b0d07c97f080405374fc13e7a34"
server_key_iv: "d79526fdb9bd0e97f544efeb238f9925"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.446424 28863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:46:59.447263 28878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.447432 28863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.447531 28863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "5552630dca0648349d5f7069f706bd58"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c5a19b0d07c97f080405374fc13e7a34"
server_key_iv: "d79526fdb9bd0e97f544efeb238f9925"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.447618 28863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:59.467407 28863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:59.469058 28863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:59.469205 28863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:59.469408 28863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:59.469681 28863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:59.469732 28863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.469771 28863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:59.469799 28863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.475553 28863 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:44031
I20260926 22:46:59.475586 28991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:44031 every 8 connection(s)
I20260926 22:46:59.475883 28863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:59.484046 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.476557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:39761 (local address 127.14.184.254:38729)
0926 22:46:59.476598 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.476600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.476796 (+ 196us) server_negotiation.cc:407] Connection header received
0926 22:46:59.477231 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.477232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.477240 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.477294 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:59.478108 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.478601 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.479551 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.479739 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.479792 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.481610 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:59.481612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:59.481612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:59.481618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:59.482992 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.483311 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.483312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.483312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.483359 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.483554 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.483555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.483555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.483648 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:59.483696 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.483918 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.483979 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:46:59.484186 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28863
I20260926 22:46:59.484268 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.476524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:39761)
0926 22:46:59.476668 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.476707 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.477154 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.477343 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.477346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.477867 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:59.478039 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.478043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.478720 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.478721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.479480 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.479482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.479819 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.479820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.479912 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.480294 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:59.480302 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:59.481530 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:59.483054 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:59.483059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:59.483060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:59.483252 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:59.483416 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:59.483417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:59.483417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:59.483504 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:59.483700 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:59.483704 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:59.483814 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.483892 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.484031 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260926 22:46:59.484294 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:46:59.484516 15075 external_mini_cluster.cc:1428] Setting key ef8bb1272de355222e2f1d65eb14501e
I20260926 22:46:59.484808 28992 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38729
I20260926 22:46:59.484905 28992 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:59.485069 28992 heartbeater.cc:511] Master 127.14.184.254:38729 requested a full tablet report, sending...
I20260926 22:46:59.485601 28801 ts_manager.cc:194] Registered new tserver with Master: 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031)
I20260926 22:46:59.486183 28801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:39761
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:46:59.507838 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38729
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:59.586756 28997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:59.586947 28997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:59.586970 28997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:59.588153 28997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:59.588203 28997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:59.588230 28997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:46:59.589663 28997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:59.589890 28997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:59.590093 28997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:59.595007 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.594997 29002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.595000 29005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:59.595168 28997 server_base.cc:1034] running on GCE node
I20260926 22:46:59.595438 28997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:59.595679 28997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:59.596850 28997 hybrid_clock.cc:648] HybridClock initialized: now 1790462819596834 us; error 31 us; skew 500 ppm
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:59.599372 28997 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:46:59.599936 28997 webserver.cc:460] Webserver started at http://127.14.184.194:41153/ using document root <none> and password file <none>
I20260926 22:46:59.600173 28997 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:59.600219 28997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:59.600366 28997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:59.601441 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "26ca139c7b3b4e849ed7182136a752a0"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "1e314dc0d41247cb221233b92892593c"
server_key_iv: "50101d67ff79faabff7f4c3654f3e77d"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.601799 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "26ca139c7b3b4e849ed7182136a752a0"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "1e314dc0d41247cb221233b92892593c"
server_key_iv: "50101d67ff79faabff7f4c3654f3e77d"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.603119 28997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:46:59.603830 29012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.604020 28997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:46:59.604117 28997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "26ca139c7b3b4e849ed7182136a752a0"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "1e314dc0d41247cb221233b92892593c"
server_key_iv: "50101d67ff79faabff7f4c3654f3e77d"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.604189 28997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:59.612336 28997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:59.614074 28997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:59.614228 28997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:59.614440 28997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:59.614820 28997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:59.614874 28997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.614915 28997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:59.614941 28997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.620715 28997 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:33867
I20260926 22:46:59.620746 29125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:33867 every 8 connection(s)
I20260926 22:46:59.621061 28997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:46:59.622128 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 28997
I20260926 22:46:59.622242 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:46:59.622408 15075 external_mini_cluster.cc:1428] Setting key 341b67eafe386de10838199302b87316
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
I20260926 22:46:59.631100 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.621783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:48505 (local address 127.14.184.254:38729)
0926 22:46:59.621840 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.621847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.622057 (+ 210us) server_negotiation.cc:407] Connection header received
0926 22:46:59.622709 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.622710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.622724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.622790 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:59.624089 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.624569 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.626066 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.626241 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.626290 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.628391 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:59.628393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:59.628393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:59.628400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:59.629708 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.630107 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.630108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.630109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.630162 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.630420 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.630421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.630421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.630529 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:59.630624 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.630922 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.631002 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 22:46:59.631336 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.621758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:48505)
0926 22:46:59.621924 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.621966 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.622570 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.622908 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.622911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.623726 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:59.623994 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.624000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.624663 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.624665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.625982 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.625985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.626319 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.626320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.626470 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.627019 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:59.627027 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:59.628300 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:59.629803 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:59.629810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:59.629812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:59.630025 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:59.630229 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:59.630230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:59.630230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:59.630360 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:59.630614 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:59.630620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:59.630788 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.630895 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.631053 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260926 22:46:59.631791 29126 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38729
I20260926 22:46:59.631863 29126 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:59.631983 29126 heartbeater.cc:511] Master 127.14.184.254:38729 requested a full tablet report, sending...
I20260926 22:46:59.632351 28801 ts_manager.cc:194] Registered new tserver with Master: 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194:33867)
I20260926 22:46:59.632694 28801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:48505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:46:59.645442 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38729
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:46:59.726655 29131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:46:59.726871 29131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:46:59.726894 29131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:46:59.728036 29131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:46:59.728078 29131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:46:59.728104 29131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:46:59.729534 29131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:46:59.729764 29131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:46:59.729986 29131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:46:59.734865 29136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:46:59.734874 29137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:59.735080 29131 server_base.cc:1034] running on GCE node
W20260926 22:46:59.734882 29139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:46:59.735337 29131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:46:59.735569 29131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:46:59.736706 29131 hybrid_clock.cc:648] HybridClock initialized: now 1790462819736695 us; error 30 us; skew 500 ppm
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:46:59.739185 29131 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:46:59.739586 29131 webserver.cc:460] Webserver started at http://127.14.184.195:34439/ using document root <none> and password file <none>
I20260926 22:46:59.739801 29131 fs_manager.cc:359] Metadata directory not provided
I20260926 22:46:59.739868 29131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:46:59.740020 29131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:46:59.740994 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "9c1d6efe13704c0e94719f67aec74e91"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c69856d25e21f5239b2c4369b86c6172"
server_key_iv: "e05b09261559e9ae90babf1f366268d8"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.741339 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "9c1d6efe13704c0e94719f67aec74e91"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c69856d25e21f5239b2c4369b86c6172"
server_key_iv: "e05b09261559e9ae90babf1f366268d8"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.742591 29131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:46:59.743346 29146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.743511 29131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:46:59.743607 29131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "9c1d6efe13704c0e94719f67aec74e91"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c69856d25e21f5239b2c4369b86c6172"
server_key_iv: "e05b09261559e9ae90babf1f366268d8"
server_key_version: "encryptionkey@0"
I20260926 22:46:59.743685 29131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:46:59.779609 29131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:46:59.781978 29131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:46:59.782164 29131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:46:59.782411 29131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:46:59.782763 29131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:46:59.782836 29131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.782878 29131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:46:59.782907 29131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:46:59.788725 29131 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33823
I20260926 22:46:59.788790 29259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33823 every 8 connection(s)
I20260926 22:46:59.789126 29131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:46:59.790050 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29131
I20260926 22:46:59.790144 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:46:59.790318 15075 external_mini_cluster.cc:1428] Setting key ecb27cf8740bdf09b106694392464b58
Sep 26 22:46:59 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462819, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:46:59.798234 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.789824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:55989 (local address 127.14.184.254:38729)
0926 22:46:59.789867 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.789870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.790092 (+ 222us) server_negotiation.cc:407] Connection header received
0926 22:46:59.790813 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.790814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.790826 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.790880 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:59.792012 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.792507 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.793504 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.793711 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.793793 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.795988 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:59.795989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:59.795989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:59.795996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:59.797128 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.797478 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.797479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.797479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.797527 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.797728 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.797729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.797729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.797825 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:59.797873 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.798100 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.798158 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:46:59.798389 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.789787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:55989)
0926 22:46:59.789934 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.789985 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.790666 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.790953 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.790957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.791717 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:46:59.791908 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.791913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.792604 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.792605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.793418 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.793420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.793802 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.793803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.793932 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.794468 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:46:59.794476 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:46:59.795905 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:46:59.797196 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:59.797201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:59.797203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:59.797415 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:59.797584 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:46:59.797585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:46:59.797585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:46:59.797675 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:46:59.797877 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:46:59.797881 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:46:59.797993 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.798071 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.798182 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260926 22:46:59.799000 29260 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38729
I20260926 22:46:59.799104 29260 heartbeater.cc:464] Registering TS with master...
I20260926 22:46:59.799299 29260 heartbeater.cc:511] Master 127.14.184.254:38729 requested a full tablet report, sending...
I20260926 22:46:59.799714 28801 ts_manager.cc:194] Registered new tserver with Master: 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823)
I20260926 22:46:59.800190 28801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:55989
I20260926 22:46:59.802112 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:46:59.809067 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.803512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.14.184.254:38729)
0926 22:46:59.803570 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.803572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.803681 (+ 109us) server_negotiation.cc:407] Connection header received
0926 22:46:59.803720 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.803721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.803731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.803783 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:46:59.804308 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.804764 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.805463 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.805631 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.805678 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.806903 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:46:59.806904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:46:59.806904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:46:59.806910 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:46:59.808113 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.808413 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.808414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.808415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.808475 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:46:59.808641 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:46:59.808642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:46:59.808642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:46:59.808782 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:46:59.808858 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.808948 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.809006 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:46:59.810581 28801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:46:59.812161 28801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:46:59.818830 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.814465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.0.0.1:35886)
0926 22:46:59.814782 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.814815 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.814866 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.815078 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.815081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.815084 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:59.815151 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.815153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.816467 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.816468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.817601 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.817603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.818469 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.818470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.818555 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.818568 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.818637 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.818740 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":48,"thread_start_us":50,"threads_started":1}
I20260926 22:46:59.818953 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.814559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.14.184.194:33867)
0926 22:46:59.815442 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.815445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.815453 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:46:59.815482 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.815483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.815494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.815582 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:59.815689 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.816544 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.817920 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.818581 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.818665 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.818747 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.818829 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.818867 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":139,"threads_started":1}
I20260926 22:46:59.819016 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.814459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33752 (local address 127.14.184.195:33823)
0926 22:46:59.814919 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.814923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.814931 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:46:59.814960 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.814962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.814973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.815135 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:59.815359 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.816430 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.817943 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.818453 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.818522 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.818579 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.818872 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.818933 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":85,"threads_started":1}
I20260926 22:46:59.819358 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.814654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.14.184.193:44031)
0926 22:46:59.814897 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.814902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.814913 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:46:59.814953 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.814955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.814969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.815064 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:59.815209 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.816359 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.817688 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.818374 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.818452 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.818508 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.819266 (+ 758us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.819301 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":135,"threads_started":1}
I20260926 22:46:59.818939 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.814329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:33867 (local address 127.0.0.1:38808)
0926 22:46:59.814493 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.814534 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.814600 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.815580 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.815581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.815583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:59.815620 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.815622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.816643 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.816644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.817830 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.817833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.818664 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.818688 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.818750 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.818755 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.818815 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.818860 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":58,"thread_start_us":65,"threads_started":1}
I20260926 22:46:59.818830 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.814329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33823 (local address 127.0.0.1:33752)
0926 22:46:59.814648 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.814659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.814764 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.815202 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.815204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.815205 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:59.815274 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.815275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.816510 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.816511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.817831 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.817833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.818611 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.818611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.818667 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.818670 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.818740 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.818771 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":54,"thread_start_us":52,"threads_started":1}
I20260926 22:46:59.820269 29060 tablet_service.cc:1467] Processing CreateTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df (DEFAULT_TABLE table=test-table [id=d3166b4011074984bde07fedb2b07eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:59.820305 28926 tablet_service.cc:1467] Processing CreateTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df (DEFAULT_TABLE table=test-table [id=d3166b4011074984bde07fedb2b07eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:59.820406 29194 tablet_service.cc:1467] Processing CreateTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df (DEFAULT_TABLE table=test-table [id=d3166b4011074984bde07fedb2b07eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:46:59.820647 29194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7372e6eead4e486d8dc017cb6b0c21df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:59.820636 28926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7372e6eead4e486d8dc017cb6b0c21df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:59.820636 29060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7372e6eead4e486d8dc017cb6b0c21df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:46:59.823212 29274 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Bootstrap starting.
I20260926 22:46:59.823827 29275 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Bootstrap starting.
I20260926 22:46:59.823855 29274 tablet_bootstrap.cc:654] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:59.824448 29274 log.cc:824] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:59.824501 29275 tablet_bootstrap.cc:654] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:59.825006 29275 log.cc:824] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:59.825253 29274 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: No bootstrap required, opened a new log
I20260926 22:46:59.825326 29274 ts_tablet_manager.cc:1397] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:46:59.825346 29276 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Bootstrap starting.
I20260926 22:46:59.825659 29275 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: No bootstrap required, opened a new log
I20260926 22:46:59.825738 29275 ts_tablet_manager.cc:1397] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:46:59.825945 29276 tablet_bootstrap.cc:654] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Neither blocks nor log segments found. Creating new log.
I20260926 22:46:59.826498 29276 log.cc:824] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Log is configured to *not* fsync() on all Append() calls
I20260926 22:46:59.827359 29276 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: No bootstrap required, opened a new log
I20260926 22:46:59.827428 29276 ts_tablet_manager.cc:1397] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:46:59.827618 29275 raft_consensus.cc:348] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.827766 29275 raft_consensus.cc:374] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:59.827826 29275 raft_consensus.cc:729] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ca139c7b3b4e849ed7182136a752a0, State: Initialized, Role: FOLLOWER
I20260926 22:46:59.827754 29274 raft_consensus.cc:348] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.827971 29274 raft_consensus.cc:374] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:59.827948 29275 consensus_queue.cc:260] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.828037 29274 raft_consensus.cc:729] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c1d6efe13704c0e94719f67aec74e91, State: Initialized, Role: FOLLOWER
I20260926 22:46:59.828156 29274 consensus_queue.cc:260] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.828245 29275 ts_tablet_manager.cc:1428] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 22:46:59.828392 29126 heartbeater.cc:503] Master 127.14.184.254:38729 was elected leader, sending a full tablet report...
I20260926 22:46:59.828410 29274 ts_tablet_manager.cc:1428] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:46:59.828540 29260 heartbeater.cc:503] Master 127.14.184.254:38729 was elected leader, sending a full tablet report...
I20260926 22:46:59.829545 29276 raft_consensus.cc:348] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.829698 29276 raft_consensus.cc:374] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:46:59.829742 29276 raft_consensus.cc:729] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5552630dca0648349d5f7069f706bd58, State: Initialized, Role: FOLLOWER
I20260926 22:46:59.829849 29276 consensus_queue.cc:260] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.830060 29276 ts_tablet_manager.cc:1428] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260926 22:46:59.830132 28992 heartbeater.cc:503] Master 127.14.184.254:38729 was elected leader, sending a full tablet report...
I20260926 22:46:59.866302 29281 raft_consensus.cc:482] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:46:59.866423 29281 raft_consensus.cc:504] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.866851 29281 leader_election.cc:288] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031), 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194:33867)
I20260926 22:46:59.869900 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.867007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.14.184.195:60537)
0926 22:46:59.867064 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.867075 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.867118 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.867306 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.867307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.867309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:59.867379 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.867381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.868151 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.868152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.868956 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.868959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.869639 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.869640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.869717 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.869721 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.869781 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.869821 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260926 22:46:59.869928 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.867103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:60537 (local address 127.14.184.193:44031)
0926 22:46:59.867174 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.867176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.867194 (+ 18us) server_negotiation.cc:407] Connection header received
0926 22:46:59.867218 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.867219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.867227 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.867277 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:59.867492 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.868069 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.869096 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.869562 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.869616 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.869623 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.869803 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.869849 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 22:46:59.870074 29283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.867012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:33867 (local address 127.14.184.195:41873)
0926 22:46:59.867368 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.867376 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:46:59.867423 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:46:59.867571 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:46:59.867572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:46:59.867573 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:46:59.867639 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.867640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.868284 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.868285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.869186 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:46:59.869188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.869807 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.869808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:46:59.869877 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.869880 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:46:59.869938 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 22:46:59.869978 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":157,"thread_start_us":118,"threads_started":1}
I20260926 22:46:59.870083 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.867077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:41873 (local address 127.14.184.194:33867)
0926 22:46:59.867150 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.867152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.867436 (+ 284us) server_negotiation.cc:407] Connection header received
0926 22:46:59.867493 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.867494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.867503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.867558 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:46:59.867711 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.868199 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.869290 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.869719 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.869790 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.869797 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.869956 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.870001 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 22:46:59.870267 28946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7372e6eead4e486d8dc017cb6b0c21df" candidate_uuid: "9c1d6efe13704c0e94719f67aec74e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5552630dca0648349d5f7069f706bd58" is_pre_election: true
I20260926 22:46:59.870406 28946 raft_consensus.cc:2454] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c1d6efe13704c0e94719f67aec74e91 in term 0.
I20260926 22:46:59.870615 29149 leader_election.cc:302] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5552630dca0648349d5f7069f706bd58, 9c1d6efe13704c0e94719f67aec74e91; no voters:
I20260926 22:46:59.870720 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7372e6eead4e486d8dc017cb6b0c21df" candidate_uuid: "9c1d6efe13704c0e94719f67aec74e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26ca139c7b3b4e849ed7182136a752a0" is_pre_election: true
I20260926 22:46:59.870769 29281 raft_consensus.cc:2781] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:46:59.870846 29281 raft_consensus.cc:482] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:46:59.870877 29281 raft_consensus.cc:3037] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:59.870872 29080 raft_consensus.cc:2454] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c1d6efe13704c0e94719f67aec74e91 in term 0.
W20260926 22:46:59.871603 29127 tablet.cc:2369] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:46:59.871677 29281 raft_consensus.cc:504] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.871800 29281 leader_election.cc:288] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [CANDIDATE]: Term 1 election: Requested vote from peers 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031), 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194:33867)
I20260926 22:46:59.871958 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7372e6eead4e486d8dc017cb6b0c21df" candidate_uuid: "9c1d6efe13704c0e94719f67aec74e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26ca139c7b3b4e849ed7182136a752a0"
I20260926 22:46:59.871945 28946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7372e6eead4e486d8dc017cb6b0c21df" candidate_uuid: "9c1d6efe13704c0e94719f67aec74e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5552630dca0648349d5f7069f706bd58"
I20260926 22:46:59.872021 29080 raft_consensus.cc:3037] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:59.872021 28946 raft_consensus.cc:3037] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:46:59.872910 29080 raft_consensus.cc:2454] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c1d6efe13704c0e94719f67aec74e91 in term 1.
I20260926 22:46:59.872905 28946 raft_consensus.cc:2454] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c1d6efe13704c0e94719f67aec74e91 in term 1.
I20260926 22:46:59.873070 29150 leader_election.cc:302] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26ca139c7b3b4e849ed7182136a752a0, 9c1d6efe13704c0e94719f67aec74e91; no voters:
I20260926 22:46:59.873220 29281 raft_consensus.cc:2781] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:46:59.873377 29281 raft_consensus.cc:686] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 1 LEADER]: Becoming Leader. State: Replica: 9c1d6efe13704c0e94719f67aec74e91, State: Running, Role: LEADER
I20260926 22:46:59.873489 29281 consensus_queue.cc:237] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:46:59.874308 28800 catalog_manager.cc:5495] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195). New cstate: current_term: 1 leader_uuid: "9c1d6efe13704c0e94719f67aec74e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } health_report { overall_health: UNKNOWN } } }
I20260926 22:46:59.891512 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.889343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.14.184.195:33823)
0926 22:46:59.889391 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:46:59.889393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:46:59.889401 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:46:59.889474 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:46:59.889475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:46:59.889491 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:46:59.889549 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:46:59.889655 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.890124 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.890884 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:46:59.891070 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:46:59.891126 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:46:59.891237 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:46:59.891330 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:46:59.891386 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:46:59.891419 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 22:46:59.891436 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:46:59.893272 29080 raft_consensus.cc:1261] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Refusing update from remote peer 9c1d6efe13704c0e94719f67aec74e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:59.893348 28946 raft_consensus.cc:1261] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Refusing update from remote peer 9c1d6efe13704c0e94719f67aec74e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:46:59.893640 29281 consensus_queue.cc:1035] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:59.893786 29284 consensus_queue.cc:1035] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:46:59.896346 29287 mvcc.cc:205] Tried to move back new op lower bound from 7333735710280048640 to 7333735710202601472. Current Snapshot: MvccSnapshot[applied={T|T < 7333735710280048640}]
I20260926 22:46:59.896629 29288 mvcc.cc:205] Tried to move back new op lower bound from 7333735710280048640 to 7333735710202601472. Current Snapshot: MvccSnapshot[applied={T|T < 7333735710280048640}]
I20260926 22:46:59.898454 28800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:46:59.898555 28800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:46:59.899323 28800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Sending DeleteTablet for 3 replicas of tablet 7372e6eead4e486d8dc017cb6b0c21df
I20260926 22:46:59.899683 29194 tablet_service.cc:1514] Processing DeleteTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:59 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:33752
I20260926 22:46:59.899711 28926 tablet_service.cc:1514] Processing DeleteTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:59 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:35886
I20260926 22:46:59.899722 29060 tablet_service.cc:1514] Processing DeleteTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:59 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38808
I20260926 22:46:59.899931 29299 tablet_replica.cc:330] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: stopping tablet replica
I20260926 22:46:59.899993 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28863
I20260926 22:46:59.900002 29297 tablet_replica.cc:330] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: stopping tablet replica
I20260926 22:46:59.900077 29299 raft_consensus.cc:2229] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:46:59.900112 29297 raft_consensus.cc:2229] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:46:59.900218 29299 raft_consensus.cc:2258] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:46:59.900266 29297 raft_consensus.cc:2258] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:46:59.900918 29299 ts_tablet_manager.cc:1905] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:46:59.900944 29297 ts_tablet_manager.cc:1905] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:46:59.902658 29297 ts_tablet_manager.cc:1918] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:46:59.902760 29297 log.cc:1196] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/7372e6eead4e486d8dc017cb6b0c21df
I20260926 22:46:59.902771 29299 ts_tablet_manager.cc:1918] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:46:59.902829 29299 log.cc:1196] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/7372e6eead4e486d8dc017cb6b0c21df
I20260926 22:46:59.903059 29297 ts_tablet_manager.cc:1939] T 7372e6eead4e486d8dc017cb6b0c21df P 9c1d6efe13704c0e94719f67aec74e91: Deleting consensus metadata
I20260926 22:46:59.903096 29299 ts_tablet_manager.cc:1939] T 7372e6eead4e486d8dc017cb6b0c21df P 26ca139c7b3b4e849ed7182136a752a0: Deleting consensus metadata
I20260926 22:46:59.903435 28786 catalog_manager.cc:4841] TS 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823): tablet 7372e6eead4e486d8dc017cb6b0c21df (table test-table [id=d3166b4011074984bde07fedb2b07eb5]) successfully deleted
I20260926 22:46:59.903954 28788 catalog_manager.cc:4841] TS 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194:33867): tablet 7372e6eead4e486d8dc017cb6b0c21df (table test-table [id=d3166b4011074984bde07fedb2b07eb5]) successfully deleted
W20260926 22:46:59.906519 28787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:44031 (error 108)
I20260926 22:46:59.906750 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28997
I20260926 22:46:59.907310 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:46:59.906872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.0.0.1:35894)
0926 22:46:59.907238 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0926 22:46:59.907271 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:44031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349}
W20260926 22:46:59.907524 28787 catalog_manager.cc:4584] TS 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7372e6eead4e486d8dc017cb6b0c21df: Network error: Client connection negotiation failed: client connection to 127.14.184.193:44031: connect: Connection refused (error 111)
I20260926 22:46:59.913631 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 29131
I20260926 22:46:59.917791 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 28769
I20260926 22:46:59.922834 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38729
--webserver_interface=127.14.184.254
--webserver_port=41901
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:38729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:00.000943 29301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:00.001189 29301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:00.001228 29301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:00.002391 29301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:47:00.002458 29301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:00.002480 29301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:00.002498 29301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:47:00.002517 29301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:47:00.004196 29301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:38729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:00.004458 29301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:00.004688 29301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:00.013478 29307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.013499 29308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.013482 29310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.013638 29301 server_base.cc:1034] running on GCE node
I20260926 22:47:00.014045 29301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:00.014356 29301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:00.015513 29301 hybrid_clock.cc:648] HybridClock initialized: now 1790462820015504 us; error 40 us; skew 500 ppm
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:00.023226 29301 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:47:00.023726 29301 webserver.cc:460] Webserver started at http://127.14.184.254:41901/ using document root <none> and password file <none>
I20260926 22:47:00.023979 29301 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:00.024062 29301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:00.025460 29301 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:00.026118 29317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.026314 29301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:00.026417 29301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "1083b9965d704a788eb40a7b8551d1f6"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "f036eae4d804674321aa9caff32b03c4"
server_key_iv: "dddeba0051ea0ca37211f2ec3a356fe7"
server_key_version: "encryptionkey@0"
I20260926 22:47:00.026902 29301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:00.041853 29301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:00.044162 29301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:00.044319 29301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:00.048617 29301 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:38729
I20260926 22:47:00.048667 29369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:38729 every 8 connection(s)
I20260926 22:47:00.049068 29301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:47:00.050616 29370 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:47:00.051326 29370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap starting.
I20260926 22:47:00.057242 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29301
I20260926 22:47:00.057556 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:44031
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:38729
--webserver_port=40115
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:00.059684 29370 log.cc:824] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:00.061560 29370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:00.061859 29370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap complete.
I20260926 22:47:00.064169 29370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.064425 29370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1083b9965d704a788eb40a7b8551d1f6, State: Initialized, Role: FOLLOWER
I20260926 22:47:00.064626 29370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.064725 29370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:47:00.064785 29370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:47:00.064836 29370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:47:00.065832 29370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.065953 29370 leader_election.cc:302] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1083b9965d704a788eb40a7b8551d1f6; no voters:
I20260926 22:47:00.066131 29370 leader_election.cc:288] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 22:47:00.066198 29374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:47:00.066355 29374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 LEADER]: Becoming Leader. State: Replica: 1083b9965d704a788eb40a7b8551d1f6, State: Running, Role: LEADER
I20260926 22:47:00.066402 29370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:47:00.066543 29374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.067291 29376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1083b9965d704a788eb40a7b8551d1f6. Latest consensus state: current_term: 2 leader_uuid: "1083b9965d704a788eb40a7b8551d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } } }
I20260926 22:47:00.067368 29376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:00.067519 29375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1083b9965d704a788eb40a7b8551d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } } }
I20260926 22:47:00.067613 29375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:00.067642 29384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:47:00.068367 29384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:47:00.068524 29384 catalog_manager.cc:1259] Loaded cluster ID: ebbfa6f1dc234c40ba369888b22cbaca
I20260926 22:47:00.068728 29384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:47:00.068995 29384 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 22:47:00.073640 29384 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 22:47:00.076004 29384 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:47:00.076200 29384 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Loaded TSK: 0
I20260926 22:47:00.076421 29384 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 22:47:00.139741 29372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:00.139950 29372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:00.139973 29372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:00.141113 29372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:00.141152 29372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:00.141178 29372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:47:00.142745 29372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:44031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=40115
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:00.142969 29372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:00.143203 29372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:00.149078 29400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.149266 29372 server_base.cc:1034] running on GCE node
W20260926 22:47:00.149228 29397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.149075 29398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.149585 29372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:00.149842 29372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:00.151006 29372 hybrid_clock.cc:648] HybridClock initialized: now 1790462820150981 us; error 44 us; skew 500 ppm
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:00.153486 29372 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:47:00.153985 29372 webserver.cc:460] Webserver started at http://127.14.184.193:40115/ using document root <none> and password file <none>
I20260926 22:47:00.154212 29372 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:00.154292 29372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:00.155727 29372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:00.156447 29407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.156615 29372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:00.156710 29372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "5552630dca0648349d5f7069f706bd58"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c5a19b0d07c97f080405374fc13e7a34"
server_key_iv: "d79526fdb9bd0e97f544efeb238f9925"
server_key_version: "encryptionkey@0"
I20260926 22:47:00.157042 29372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:00.178999 29372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:00.181350 29372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:00.181502 29372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:00.181787 29372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:00.182300 29414 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 22:47:00.183787 29372 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 22:47:00.183837 29372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 22:47:00.183873 29372 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 22:47:00.184537 29372 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 22:47:00.184582 29372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:00.184624 29414 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Bootstrap starting.
I20260926 22:47:00.192909 29372 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:44031
I20260926 22:47:00.193418 29372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:47:00.194125 29521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:44031 every 8 connection(s)
I20260926 22:47:00.196329 29414 log.cc:824] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:00.198837 29414 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:00.199146 29414 tablet_bootstrap.cc:492] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Bootstrap complete.
I20260926 22:47:00.200315 29414 ts_tablet_manager.cc:1397] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260926 22:47:00.201239 29414 raft_consensus.cc:348] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:47:00.201481 29414 raft_consensus.cc:729] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5552630dca0648349d5f7069f706bd58, State: Initialized, Role: FOLLOWER
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:00.201644 29414 consensus_queue.cc:260] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } } peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } }
I20260926 22:47:00.201926 29414 ts_tablet_manager.cc:1428] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260926 22:47:00.202646 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29372
I20260926 22:47:00.202999 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:33867
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:38729
--webserver_port=41153
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:00.205821 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.194521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:36271 (local address 127.14.184.254:38729)
0926 22:47:00.195429 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.195438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.195470 (+ 32us) server_negotiation.cc:407] Connection header received
0926 22:47:00.195594 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.195601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.195650 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.195775 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:00.196640 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.197924 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.199101 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.199733 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.199847 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.202114 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:00.202117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:00.202118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:00.202127 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:00.204157 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.204567 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.204570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.204571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.204666 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.204931 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.204932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.204932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.205144 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:00.205227 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.205560 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.205665 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":98,"threads_started":1}
I20260926 22:47:00.205914 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.194148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:36271)
0926 22:47:00.194500 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.194544 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.195231 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.195848 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.195853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.196267 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:00.196537 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.196543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.198057 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.198058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.199024 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.199027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.199834 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.199835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.199926 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.200540 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:00.200548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:00.202020 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:00.204237 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.204244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.204246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.204469 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.204736 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.204737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.204737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.204848 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.205218 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:00.205225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:00.205389 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.205496 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.205671 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":96,"threads_started":1}
I20260926 22:47:00.206619 29522 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38729
I20260926 22:47:00.206738 29522 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:00.206929 29522 heartbeater.cc:511] Master 127.14.184.254:38729 requested a full tablet report, sending...
I20260926 22:47:00.207850 29334 ts_manager.cc:194] Registered new tserver with Master: 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031)
I20260926 22:47:00.209143 29334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:36271
I20260926 22:47:00.210331 29522 heartbeater.cc:503] Master 127.14.184.254:38729 was elected leader, sending a full tablet report...
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:47:00.216276 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.208571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.14.184.193:44031)
0926 22:47:00.208780 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.208785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.208796 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:47:00.208860 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.208862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.208876 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.208936 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:00.209352 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.210150 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.211146 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.211426 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.211504 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.213429 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:00.213436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:00.213437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:00.213443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:00.214990 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.215393 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.215396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.215396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.215455 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.215659 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.215661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.215661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.215819 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:00.215912 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.216094 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.216181 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":63,"threads_started":1}
I20260926 22:47:00.216286 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.208522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.0.0.1:35904)
0926 22:47:00.208683 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.208717 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.208783 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.208936 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.208939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.209182 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:00.209258 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.209260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.210238 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.210239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.211054 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.211056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.211516 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.211517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.211591 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.212069 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:00.212073 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:00.213338 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:00.215087 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.215093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.215094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.215296 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.215533 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.215534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.215534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.215597 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.215902 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:00.215908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:00.215959 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.216060 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.216182 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
I20260926 22:47:00.216688 29453 tablet_service.cc:1514] Processing DeleteTablet for tablet 7372e6eead4e486d8dc017cb6b0c21df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:46:59 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:35904
I20260926 22:47:00.217200 29531 tablet_replica.cc:330] stopping tablet replica
I20260926 22:47:00.217326 29531 raft_consensus.cc:2229] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:00.217398 29531 raft_consensus.cc:2258] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:00.217689 29531 ts_tablet_manager.cc:1905] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:47:00.219650 29531 ts_tablet_manager.cc:1918] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:47:00.219710 29531 log.cc:1196] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/7372e6eead4e486d8dc017cb6b0c21df
I20260926 22:47:00.219940 29531 ts_tablet_manager.cc:1939] T 7372e6eead4e486d8dc017cb6b0c21df P 5552630dca0648349d5f7069f706bd58: Deleting consensus metadata
I20260926 22:47:00.220417 29320 catalog_manager.cc:4841] TS 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031): tablet 7372e6eead4e486d8dc017cb6b0c21df (table test-table [id=d3166b4011074984bde07fedb2b07eb5]) successfully deleted
W20260926 22:47:00.285490 29528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:00.285732 29528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:00.285771 29528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:00.286945 29528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:00.287015 29528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:00.287057 29528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:47:00.288511 29528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:33867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=41153
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:00.288769 29528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:00.288990 29528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:00.293865 29536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.293883 29537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.293867 29539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.294066 29528 server_base.cc:1034] running on GCE node
I20260926 22:47:00.294275 29528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:00.294554 29528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:00.295722 29528 hybrid_clock.cc:648] HybridClock initialized: now 1790462820295702 us; error 33 us; skew 500 ppm
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:00.298173 29528 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:47:00.298593 29528 webserver.cc:460] Webserver started at http://127.14.184.194:41153/ using document root <none> and password file <none>
I20260926 22:47:00.298834 29528 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:00.298892 29528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:00.300133 29528 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:00.300650 29546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.300830 29528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:00.300925 29528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "26ca139c7b3b4e849ed7182136a752a0"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "1e314dc0d41247cb221233b92892593c"
server_key_iv: "50101d67ff79faabff7f4c3654f3e77d"
server_key_version: "encryptionkey@0"
I20260926 22:47:00.301249 29528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:00.310638 29528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:00.312808 29528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:00.312961 29528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:00.313167 29528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:00.313452 29528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:00.313504 29528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.313544 29528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:00.313571 29528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.319553 29528 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:33867
I20260926 22:47:00.319605 29659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:33867 every 8 connection(s)
I20260926 22:47:00.319908 29528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:00.328651 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29528
I20260926 22:47:00.328771 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.320554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:48663 (local address 127.14.184.254:38729)
0926 22:47:00.320646 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.320648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.320791 (+ 143us) server_negotiation.cc:407] Connection header received
0926 22:47:00.321286 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.321287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.321298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.321380 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:00.322240 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.322851 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.323759 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.323962 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.324015 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.326011 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:00.326012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:00.326013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:00.326019 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:00.327398 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.327785 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.327786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.327786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.327847 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.328096 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.328097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.328098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.328223 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:00.328302 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.328568 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.328662 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 22:47:00.328910 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.320514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:48663)
0926 22:47:00.320665 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.320698 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.321138 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.321435 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.321439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.321970 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:00.322146 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.322150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.322924 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.322925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.323688 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.323690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.324032 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.324032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.324138 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.324637 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:00.324645 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:00.325926 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:00.327470 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.327476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.327477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.327682 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.327921 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.327922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.327922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.328015 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.328300 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:00.328304 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:00.328420 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.328502 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.328617 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":94,"threads_started":1}
I20260926 22:47:00.329012 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:33823
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:38729
--webserver_port=34439
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:00.329402 29660 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38729
I20260926 22:47:00.329481 29660 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:00.329674 29660 heartbeater.cc:511] Master 127.14.184.254:38729 requested a full tablet report, sending...
I20260926 22:47:00.330116 29334 ts_manager.cc:194] Registered new tserver with Master: 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194:33867)
I20260926 22:47:00.330554 29334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:48663
W20260926 22:47:00.408722 29663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:00.408911 29663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:00.408941 29663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:00.410323 29663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:00.410362 29663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:00.410388 29663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:47:00.411953 29663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:33823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=34439
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:00.412187 29663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:00.412407 29663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:00.417207 29669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.417272 29668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.417223 29671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.417428 29663 server_base.cc:1034] running on GCE node
I20260926 22:47:00.417740 29663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:00.417972 29663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:00.419104 29663 hybrid_clock.cc:648] HybridClock initialized: now 1790462820419093 us; error 27 us; skew 500 ppm
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:00.421422 29663 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:47:00.421820 29663 webserver.cc:460] Webserver started at http://127.14.184.195:34439/ using document root <none> and password file <none>
I20260926 22:47:00.422039 29663 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:00.422117 29663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:00.423480 29663 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:47:00.424065 29678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.424274 29663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:47:00.424366 29663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "9c1d6efe13704c0e94719f67aec74e91"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c69856d25e21f5239b2c4369b86c6172"
server_key_iv: "e05b09261559e9ae90babf1f366268d8"
server_key_version: "encryptionkey@0"
I20260926 22:47:00.424674 29663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:00.450896 29663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:00.453187 29663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:00.453344 29663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:00.453581 29663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:00.453953 29663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:00.454005 29663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.454051 29663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:00.454078 29663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.460523 29663 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33823
I20260926 22:47:00.460629 29791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33823 every 8 connection(s)
I20260926 22:47:00.461002 29663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:47:00.464063 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29663
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:00.472342 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.461772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:59889 (local address 127.14.184.254:38729)
0926 22:47:00.461816 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.461818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.462117 (+ 299us) server_negotiation.cc:407] Connection header received
0926 22:47:00.463396 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.463397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.463412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.463504 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:00.464327 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.465094 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.466595 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.466930 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.467000 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.469380 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:00.469381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:00.469381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:00.469395 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:00.470988 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.471392 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.471394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.471394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.471449 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.471705 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.471706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.471706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.471822 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:00.471892 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.472176 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.472253 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:47:00.472538 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.461785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:59889)
0926 22:47:00.461981 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.462043 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.463256 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.463583 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.463588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.463946 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:00.464242 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.464248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.465201 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.465202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.466528 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.466530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.467016 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.467016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.467164 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.467612 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:00.467623 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:00.469286 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:00.471066 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.471072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.471074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.471295 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.471520 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:00.471521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:00.471521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:00.471636 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:00.471902 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:00.471908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:00.472042 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.472141 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.472295 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260926 22:47:00.472776 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.465278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.14.184.254:38729)
0926 22:47:00.465706 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.465708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.465717 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:47:00.465738 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.465739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.465750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.465814 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:00.466517 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.467269 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.468350 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.468658 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.468738 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.470111 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:00.470113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:00.470113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:00.470120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:00.471517 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.471874 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.471876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.471876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.471933 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:00.472133 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:00.472134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:00.472134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:00.472243 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:00.472423 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.472461 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.472536 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":78,"threads_started":1}
I20260926 22:47:00.473168 29792 heartbeater.cc:347] Connected to a master server at 127.14.184.254:38729
I20260926 22:47:00.473291 29792 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:00.473470 29792 heartbeater.cc:511] Master 127.14.184.254:38729 requested a full tablet report, sending...
I20260926 22:47:00.473848 29334 ts_manager.cc:194] Registered new tserver with Master: 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823)
I20260926 22:47:00.474211 29334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:59889
I20260926 22:47:00.484297 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:47:00.487213 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.484659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.14.184.254:38729)
0926 22:47:00.484734 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.484736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.484842 (+ 106us) server_negotiation.cc:407] Connection header received
0926 22:47:00.484928 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.484929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.484969 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.485054 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:47:00.485157 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.485624 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.486388 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.486596 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.486673 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.486892 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:47:00.486999 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:47:00.487096 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.487116 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.487133 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 22:47:00.487485 29334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:47:00.488991 29334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:47:00.491241 29453 tablet_service.cc:1467] Processing CreateTablet for tablet f4c692b237dc4d47978530898fb43769 (DEFAULT_TABLE table=test-table [id=c1689e6bab264cb6b06cc4563f1314f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:00.491452 29453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c692b237dc4d47978530898fb43769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:00.493067 29414 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Bootstrap starting.
I20260926 22:47:00.493409 29414 tablet_bootstrap.cc:654] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:00.493940 29414 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: No bootstrap required, opened a new log
I20260926 22:47:00.494001 29414 ts_tablet_manager.cc:1397] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:00.494366 29414 raft_consensus.cc:348] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.494431 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.490945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:33867 (local address 127.0.0.1:38824)
0926 22:47:00.490992 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.491004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.491041 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.491383 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.491384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.491386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:00.491433 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.491435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.492398 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.492399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.493242 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.493245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.494174 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.494175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.494260 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.494284 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.494315 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.494363 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 22:47:00.494449 29414 raft_consensus.cc:374] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:00.494504 29414 raft_consensus.cc:729] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5552630dca0648349d5f7069f706bd58, State: Initialized, Role: FOLLOWER
I20260926 22:47:00.494557 29414 consensus_queue.cc:260] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.494632 29414 ts_tablet_manager.cc:1428] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:00.494799 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.491032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.14.184.194:33867)
0926 22:47:00.491203 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.491208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.491219 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:47:00.491268 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.491271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.491288 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.491379 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:00.491519 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.492309 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.493336 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.494070 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.494547 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.494615 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.494641 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.494721 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260926 22:47:00.495401 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.490981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33823 (local address 127.0.0.1:33768)
0926 22:47:00.491139 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.491147 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.491181 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.491543 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.491544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.491546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:00.491604 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.491605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.493067 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.493068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.494179 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.494181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.495229 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.495230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.495299 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.495304 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.495327 (+ 23us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.495357 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260926 22:47:00.495464 29594 tablet_service.cc:1467] Processing CreateTablet for tablet f4c692b237dc4d47978530898fb43769 (DEFAULT_TABLE table=test-table [id=c1689e6bab264cb6b06cc4563f1314f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:00.495751 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.491198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.14.184.195:33823)
0926 22:47:00.491363 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.491368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.491378 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:47:00.491417 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.491419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.491435 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.491680 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:00.491755 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.492934 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.494268 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.495130 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.495524 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.495584 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.495614 (+ 30us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.495662 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260926 22:47:00.495783 29594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c692b237dc4d47978530898fb43769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:00.496356 29726 tablet_service.cc:1467] Processing CreateTablet for tablet f4c692b237dc4d47978530898fb43769 (DEFAULT_TABLE table=test-table [id=c1689e6bab264cb6b06cc4563f1314f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:00.496698 29726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c692b237dc4d47978530898fb43769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:00.499296 29802 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Bootstrap starting.
I20260926 22:47:00.499882 29802 tablet_bootstrap.cc:654] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:00.499984 29803 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Bootstrap starting.
I20260926 22:47:00.500439 29802 log.cc:824] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:00.500494 29803 tablet_bootstrap.cc:654] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:00.500911 29803 log.cc:824] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:00.501227 29802 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: No bootstrap required, opened a new log
I20260926 22:47:00.501305 29802 ts_tablet_manager.cc:1397] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:00.501627 29803 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: No bootstrap required, opened a new log
I20260926 22:47:00.501709 29803 ts_tablet_manager.cc:1397] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:47:00.502774 29802 raft_consensus.cc:348] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.502940 29802 raft_consensus.cc:374] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:00.502988 29802 raft_consensus.cc:729] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ca139c7b3b4e849ed7182136a752a0, State: Initialized, Role: FOLLOWER
I20260926 22:47:00.503109 29802 consensus_queue.cc:260] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.503408 29802 ts_tablet_manager.cc:1428] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:00.503440 29660 heartbeater.cc:503] Master 127.14.184.254:38729 was elected leader, sending a full tablet report...
I20260926 22:47:00.503360 29803 raft_consensus.cc:348] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.503491 29803 raft_consensus.cc:374] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:00.503531 29803 raft_consensus.cc:729] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c1d6efe13704c0e94719f67aec74e91, State: Initialized, Role: FOLLOWER
I20260926 22:47:00.503648 29803 consensus_queue.cc:260] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.503922 29803 ts_tablet_manager.cc:1428] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:47:00.503947 29792 heartbeater.cc:503] Master 127.14.184.254:38729 was elected leader, sending a full tablet report...
W20260926 22:47:00.570405 29661 tablet.cc:2369] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:47:00.694648 29523 tablet.cc:2369] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:47:00.711696 29793 tablet.cc:2369] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:47:00.727240 29806 raft_consensus.cc:482] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:47:00.727396 29806 raft_consensus.cc:504] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.727841 29806 leader_election.cc:288] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823), 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031)
I20260926 22:47:00.730975 29808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.728083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:52953 (local address 127.14.184.193:44031)
0926 22:47:00.728205 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.728207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.728216 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:47:00.728245 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.728246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.728274 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.728358 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:00.728552 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.729057 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.730067 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.730579 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.730730 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.730808 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.730851 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.730901 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260926 22:47:00.730973 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.728017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.14.184.194:52953)
0926 22:47:00.728066 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.728079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.728122 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.728382 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.728383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.728386 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:00.728471 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.728473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.729125 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.729126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.729986 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.729989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.730715 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.730716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.730800 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.730803 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.730850 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.730886 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 22:47:00.731067 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.728140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:60941 (local address 127.14.184.195:33823)
0926 22:47:00.728183 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.728185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.728311 (+ 126us) server_negotiation.cc:407] Connection header received
0926 22:47:00.728371 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.728372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.728383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.728463 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:00.728621 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.729175 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.730172 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.730624 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.730778 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.730788 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.730949 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.730990 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:47:00.731148 29809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.728017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33823 (local address 127.14.184.194:60941)
0926 22:47:00.728229 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.728237 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:00.728286 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:00.728467 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:00.728468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:00.728469 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:00.728549 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.728551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.729257 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.729257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.730100 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:00.730102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.730798 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.730799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:00.730873 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.730877 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:47:00.730934 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 22:47:00.730968 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":42,"thread_start_us":93,"threads_started":1}
I20260926 22:47:00.731374 29469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c692b237dc4d47978530898fb43769" candidate_uuid: "26ca139c7b3b4e849ed7182136a752a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5552630dca0648349d5f7069f706bd58" is_pre_election: true
I20260926 22:47:00.731447 29746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c692b237dc4d47978530898fb43769" candidate_uuid: "26ca139c7b3b4e849ed7182136a752a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c1d6efe13704c0e94719f67aec74e91" is_pre_election: true
I20260926 22:47:00.731523 29469 raft_consensus.cc:2454] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26ca139c7b3b4e849ed7182136a752a0 in term 0.
I20260926 22:47:00.731565 29746 raft_consensus.cc:2454] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26ca139c7b3b4e849ed7182136a752a0 in term 0.
I20260926 22:47:00.731757 29549 leader_election.cc:302] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26ca139c7b3b4e849ed7182136a752a0, 5552630dca0648349d5f7069f706bd58; no voters:
I20260926 22:47:00.731890 29806 raft_consensus.cc:2781] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:47:00.731941 29806 raft_consensus.cc:482] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:47:00.731973 29806 raft_consensus.cc:3037] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:00.732656 29806 raft_consensus.cc:504] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.732797 29806 leader_election.cc:288] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823), 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031)
I20260926 22:47:00.733028 29469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c692b237dc4d47978530898fb43769" candidate_uuid: "26ca139c7b3b4e849ed7182136a752a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5552630dca0648349d5f7069f706bd58"
I20260926 22:47:00.733072 29746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c692b237dc4d47978530898fb43769" candidate_uuid: "26ca139c7b3b4e849ed7182136a752a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c1d6efe13704c0e94719f67aec74e91"
I20260926 22:47:00.733115 29469 raft_consensus.cc:3037] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:00.733132 29746 raft_consensus.cc:3037] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:00.733940 29469 raft_consensus.cc:2454] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26ca139c7b3b4e849ed7182136a752a0 in term 1.
I20260926 22:47:00.733994 29746 raft_consensus.cc:2454] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26ca139c7b3b4e849ed7182136a752a0 in term 1.
I20260926 22:47:00.734108 29549 leader_election.cc:302] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26ca139c7b3b4e849ed7182136a752a0, 5552630dca0648349d5f7069f706bd58; no voters:
I20260926 22:47:00.734223 29806 raft_consensus.cc:2781] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:47:00.734360 29806 raft_consensus.cc:686] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 LEADER]: Becoming Leader. State: Replica: 26ca139c7b3b4e849ed7182136a752a0, State: Running, Role: LEADER
I20260926 22:47:00.734469 29806 consensus_queue.cc:237] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:00.735316 29334 catalog_manager.cc:5495] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "26ca139c7b3b4e849ed7182136a752a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } health_report { overall_health: UNKNOWN } } }
I20260926 22:47:00.776927 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.774447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.14.184.194:33867)
0926 22:47:00.774491 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:00.774493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:00.774502 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:47:00.774582 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:00.774583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:00.774606 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:00.774730 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:47:00.774897 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.775373 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.776127 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:00.776328 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:00.776406 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:00.776563 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:47:00.776663 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:47:00.776757 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:00.776799 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 22:47:00.776821 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:47:00.778589 29469 raft_consensus.cc:1261] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Refusing update from remote peer 26ca139c7b3b4e849ed7182136a752a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:47:00.778606 29746 raft_consensus.cc:1261] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Refusing update from remote peer 26ca139c7b3b4e849ed7182136a752a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:47:00.778985 29810 consensus_queue.cc:1035] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:47:00.779115 29806 consensus_queue.cc:1035] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:47:00.781008 29813 mvcc.cc:205] Tried to move back new op lower bound from 7333735713906454528 to 7333735713728921600. Current Snapshot: MvccSnapshot[applied={T|T < 7333735713906454528}]
I20260926 22:47:00.781575 29815 mvcc.cc:205] Tried to move back new op lower bound from 7333735713906454528 to 7333735713728921600. Current Snapshot: MvccSnapshot[applied={T|T < 7333735713906454528}]
I20260926 22:47:00.781809 29814 mvcc.cc:205] Tried to move back new op lower bound from 7333735713906454528 to 7333735713728921600. Current Snapshot: MvccSnapshot[applied={T|T < 7333735713906454528}]
I20260926 22:47:00.783581 29334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:47:00.783676 29334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:47:00.784387 29334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Sending DeleteTablet for 3 replicas of tablet f4c692b237dc4d47978530898fb43769
I20260926 22:47:00.784674 29594 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c692b237dc4d47978530898fb43769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:00 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38824
I20260926 22:47:00.784745 29453 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c692b237dc4d47978530898fb43769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:00 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:35904
I20260926 22:47:00.784881 29726 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c692b237dc4d47978530898fb43769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:00 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:33768
I20260926 22:47:00.784938 29824 tablet_replica.cc:330] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: stopping tablet replica
I20260926 22:47:00.784994 29824 raft_consensus.cc:2229] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:00.785046 29824 raft_consensus.cc:2258] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:00.785142 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 29372
W20260926 22:47:00.790216 29320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:44031 (error 108)
I20260926 22:47:00.790511 29823 tablet_replica.cc:330] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: stopping tablet replica
I20260926 22:47:00.790551 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 29528
I20260926 22:47:00.790647 29823 raft_consensus.cc:2229] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:47:00.790802 29823 raft_consensus.cc:2258] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:00.790939 29825 tablet_replica.cc:330] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: stopping tablet replica
I20260926 22:47:00.791026 29825 raft_consensus.cc:2229] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:00.791112 29825 raft_consensus.cc:2258] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:00.791862 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.791735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.0.0.1:35918)
0926 22:47:00.791782 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.791817 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:44031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 22:47:00.792533 29825 ts_tablet_manager.cc:1905] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 22:47:00.792321 29320 catalog_manager.cc:4584] TS 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4c692b237dc4d47978530898fb43769: Network error: Client connection negotiation failed: client connection to 127.14.184.193:44031: connect: Connection refused (error 111)
I20260926 22:47:00.794113 29825 ts_tablet_manager.cc:1918] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:47:00.794178 29825 log.cc:1196] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/f4c692b237dc4d47978530898fb43769
I20260926 22:47:00.794463 29825 ts_tablet_manager.cc:1939] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Deleting consensus metadata
I20260926 22:47:00.794912 29319 catalog_manager.cc:4841] TS 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823): tablet f4c692b237dc4d47978530898fb43769 (table test-table [id=c1689e6bab264cb6b06cc4563f1314f9]) successfully deleted
I20260926 22:47:00.795884 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 29663
I20260926 22:47:00.795957 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:00.795851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:33867 (local address 127.0.0.1:38856)
0926 22:47:00.795898 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:00.795923 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.194:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 22:47:00.800590 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 29301
I20260926 22:47:00.805671 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38729
--webserver_interface=127.14.184.254
--webserver_port=41901
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:38729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:00.883878 29827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:00.884066 29827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:00.884087 29827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:00.885141 29827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:47:00.885176 29827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:00.885191 29827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:00.885205 29827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:47:00.885216 29827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:47:00.886567 29827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:38729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:38729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:00.886814 29827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:00.886996 29827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:00.891842 29835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:00.891846 29833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.891901 29827 server_base.cc:1034] running on GCE node
W20260926 22:47:00.891850 29832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:00.892344 29827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:00.892642 29827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:00.893810 29827 hybrid_clock.cc:648] HybridClock initialized: now 1790462820893795 us; error 30 us; skew 500 ppm
Sep 26 22:47:00 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462820, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:00.896189 29827 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:47:00.896636 29827 webserver.cc:460] Webserver started at http://127.14.184.254:41901/ using document root <none> and password file <none>
I20260926 22:47:00.896869 29827 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:00.896943 29827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:00.898227 29827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:00.899044 29842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:00.899238 29827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:00.899343 29827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "1083b9965d704a788eb40a7b8551d1f6"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "f036eae4d804674321aa9caff32b03c4"
server_key_iv: "dddeba0051ea0ca37211f2ec3a356fe7"
server_key_version: "encryptionkey@0"
I20260926 22:47:00.899675 29827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:00.909538 29827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:00.911315 29827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:00.911468 29827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:00.915206 29827 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:38729
I20260926 22:47:00.915242 29894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:38729 every 8 connection(s)
I20260926 22:47:00.915608 29827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:47:00.917060 29895 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:47:00.917725 29895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap starting.
I20260926 22:47:00.920322 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29827
I20260926 22:47:00.920648 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:44031
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:38729
--webserver_port=40115
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:00.926589 29895 log.cc:824] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:00.928756 29895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:00.929059 29895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6: Bootstrap complete.
I20260926 22:47:00.931377 29895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.931586 29895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1083b9965d704a788eb40a7b8551d1f6, State: Initialized, Role: FOLLOWER
I20260926 22:47:00.931756 29895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.931814 29895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:47:00.931885 29895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:47:00.931926 29895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 22:47:00.932863 29895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.932945 29895 leader_election.cc:302] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1083b9965d704a788eb40a7b8551d1f6; no voters:
I20260926 22:47:00.933158 29895 leader_election.cc:288] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 22:47:00.933197 29899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 22:47:00.933312 29899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [term 3 LEADER]: Becoming Leader. State: Replica: 1083b9965d704a788eb40a7b8551d1f6, State: Running, Role: LEADER
I20260926 22:47:00.933401 29899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } }
I20260926 22:47:00.933430 29895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:47:00.933662 29900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1083b9965d704a788eb40a7b8551d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } } }
I20260926 22:47:00.933861 29900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:00.933663 29899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1083b9965d704a788eb40a7b8551d1f6. Latest consensus state: current_term: 3 leader_uuid: "1083b9965d704a788eb40a7b8551d1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1083b9965d704a788eb40a7b8551d1f6" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 38729 } } }
I20260926 22:47:00.933967 29899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1083b9965d704a788eb40a7b8551d1f6 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:00.934780 29912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:47:00.935563 29912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:47:00.935667 29912 catalog_manager.cc:1259] Loaded cluster ID: ebbfa6f1dc234c40ba369888b22cbaca
I20260926 22:47:00.935710 29912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 22:47:00.935923 29912 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f43730f49ad google::LogMessage::Fail() at ??:0
@ 0x7f43730f788a google::LogMessage::SendToLog() at ??:0
@ 0x7f43730f44de google::LogMessage::Flush() at ??:0
@ 0x7f43730f5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f437b0150f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f437b01e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4374fc20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4374fb7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4375e1b6db start_thread at ??:0
@ 0x7f4371bf261f clone at ??:0
W20260926 22:47:01.031965 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:01.032229 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:01.032265 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:01.033926 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:01.033985 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:01.034024 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:47:01.036157 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:44031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=40115
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:01.036410 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:01.036654 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:01.041953 29934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:01.042114 29897 server_base.cc:1034] running on GCE node
W20260926 22:47:01.041965 29932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:01.041967 29931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:01.042418 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:01.042707 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:01.043855 29897 hybrid_clock.cc:648] HybridClock initialized: now 1790462821043832 us; error 41 us; skew 500 ppm
Sep 26 22:47:01 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462821, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:01.046295 29897 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:47:01.046798 29897 webserver.cc:460] Webserver started at http://127.14.184.193:40115/ using document root <none> and password file <none>
I20260926 22:47:01.047027 29897 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:01.047089 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:01.048416 29897 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:01.049110 29941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:01.049281 29897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:01.049367 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "5552630dca0648349d5f7069f706bd58"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c5a19b0d07c97f080405374fc13e7a34"
server_key_iv: "d79526fdb9bd0e97f544efeb238f9925"
server_key_version: "encryptionkey@0"
I20260926 22:47:01.049696 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:01.090847 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:01.092969 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:01.093133 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:01.093362 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:01.093822 29948 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 22:47:01.095885 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 22:47:01.095943 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 22:47:01.095986 29897 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 22:47:01.096621 29897 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 22:47:01.096674 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:01.096751 29948 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Bootstrap starting.
I20260926 22:47:01.103659 29897 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:44031
I20260926 22:47:01.104063 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:47:01.104538 30055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:44031 every 8 connection(s)
I20260926 22:47:01.106370 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 29897
I20260926 22:47:01.106627 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:33867
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:38729
--webserver_port=41153
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:01.107863 29948 log.cc:824] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:01.110126 29948 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:01.110416 29948 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Bootstrap complete.
I20260926 22:47:01.111680 29948 ts_tablet_manager.cc:1397] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20260926 22:47:01.113445 29948 raft_consensus.cc:348] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:01.113835 29948 raft_consensus.cc:729] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5552630dca0648349d5f7069f706bd58, State: Initialized, Role: FOLLOWER
I20260926 22:47:01.113983 29948 consensus_queue.cc:260] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:01.114290 29948 ts_tablet_manager.cc:1428] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
W20260926 22:47:01.139837 30058 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:47:01.104813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:59113)
0926 22:47:01.105007 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.105052 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:01.105481 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:01.139639 (+ 34158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
W20260926 22:47:01.140127 30056 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:38729 (0 consecutive failures): Network error: Failed to ping master at 127.14.184.254:38729: Client connection negotiation failed: client connection to 127.14.184.254:38729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 22:47:01.140486 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.140372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:60401)
0926 22:47:01.140424 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.140467 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 22:47:01.140830 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.140748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:50149)
0926 22:47:01.140793 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.140815 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 22:47:01.140955 30056 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 22:47:01.190984 30059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:01.191248 30059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:01.191287 30059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:01.192706 30059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:01.192756 30059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:01.192791 30059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:47:01.194216 30059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:33867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=41153
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:01.194380 30059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:01.194545 30059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:01.199610 30066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:01.199621 30069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:01.199630 30067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:01.199844 30059 server_base.cc:1034] running on GCE node
I20260926 22:47:01.200064 30059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:01.200325 30059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:01.201500 30059 hybrid_clock.cc:648] HybridClock initialized: now 1790462821201481 us; error 33 us; skew 500 ppm
Sep 26 22:47:01 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462821, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:01.204327 30059 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:47:01.204730 30059 webserver.cc:460] Webserver started at http://127.14.184.194:41153/ using document root <none> and password file <none>
I20260926 22:47:01.204914 30059 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:01.204957 30059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:01.206354 30059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:01.207120 30076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:01.207304 30059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:01.207398 30059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "26ca139c7b3b4e849ed7182136a752a0"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "1e314dc0d41247cb221233b92892593c"
server_key_iv: "50101d67ff79faabff7f4c3654f3e77d"
server_key_version: "encryptionkey@0"
I20260926 22:47:01.207759 30059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:01.223024 30059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:01.225466 30059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:01.225576 30059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:01.225816 30059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:01.226284 30083 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 22:47:01.227734 30059 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 22:47:01.227804 30059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 22:47:01.227852 30059 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 22:47:01.228536 30059 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 22:47:01.228595 30059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:01.228621 30083 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Bootstrap starting.
I20260926 22:47:01.235860 30059 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:33867
I20260926 22:47:01.235924 30190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:33867 every 8 connection(s)
I20260926 22:47:01.236272 30059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:47:01.237524 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.237019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:41333)
0926 22:47:01.237220 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.237331 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
W20260926 22:47:01.237751 30191 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:38729 (0 consecutive failures): Network error: Failed to ping master at 127.14.184.254:38729: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
I20260926 22:47:01.238065 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.237938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:38367)
0926 22:47:01.237990 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.238027 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 22:47:01.238420 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.238305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:40673)
0926 22:47:01.238362 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.238388 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 22:47:01.238543 30191 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 22:47:01.238824 30083 log.cc:824] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:01.240820 30083 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:01.241048 30083 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Bootstrap complete.
I20260926 22:47:01.241931 30083 ts_tablet_manager.cc:1397] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260926 22:47:01.242842 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30059
I20260926 22:47:01.243140 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:33823
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:38729
--webserver_port=34439
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:01.243330 30083 raft_consensus.cc:348] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:01.243619 30083 raft_consensus.cc:729] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ca139c7b3b4e849ed7182136a752a0, State: Initialized, Role: FOLLOWER
I20260926 22:47:01.243762 30083 consensus_queue.cc:260] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:01.244042 30083 ts_tablet_manager.cc:1428] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260926 22:47:01.244338 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.244222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:40727)
0926 22:47:01.244277 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.244307 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 22:47:01.244750 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.244618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:40199)
0926 22:47:01.244679 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.244727 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 22:47:01.323288 30196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:01.323508 30196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:01.323565 30196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:01.324676 30196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:01.324750 30196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:01.324795 30196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:47:01.326243 30196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:33823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=34439
--tserver_master_addrs=127.14.184.254:38729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:01.326494 30196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:01.326738 30196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:01.331638 30202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:01.331609 30201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:01.331835 30196 server_base.cc:1034] running on GCE node
W20260926 22:47:01.331620 30204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:01.332120 30196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:01.332355 30196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:01.333493 30196 hybrid_clock.cc:648] HybridClock initialized: now 1790462821333474 us; error 36 us; skew 500 ppm
Sep 26 22:47:01 dist-test-slave-3986 krb5kdc[28756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462821, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:01.335906 30196 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:47:01.336346 30196 webserver.cc:460] Webserver started at http://127.14.184.195:34439/ using document root <none> and password file <none>
I20260926 22:47:01.336567 30196 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:01.336633 30196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:01.337949 30196 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:01.338533 30211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:01.338744 30196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:01.338840 30196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "9c1d6efe13704c0e94719f67aec74e91"
format_stamp: "Formatted at 2026-09-26 22:46:59 on dist-test-slave-3986"
server_key: "c69856d25e21f5239b2c4369b86c6172"
server_key_iv: "e05b09261559e9ae90babf1f366268d8"
server_key_version: "encryptionkey@0"
I20260926 22:47:01.339161 30196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:01.354630 30196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:01.356856 30196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:01.357016 30196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:01.357230 30196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:01.357525 30196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:01.357579 30196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:01.357611 30196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:01.357640 30196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:01.363361 30196 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33823
I20260926 22:47:01.363397 30324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33823 every 8 connection(s)
I20260926 22:47:01.363722 30196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:47:01.364717 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.364369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:37573)
0926 22:47:01.364514 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.364583 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
W20260926 22:47:01.365097 30325 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:38729 (0 consecutive failures): Network error: Failed to ping master at 127.14.184.254:38729: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
I20260926 22:47:01.365339 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.365256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:48889)
0926 22:47:01.365299 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.365324 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 22:47:01.365594 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:01.365526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:49757)
0926 22:47:01.365562 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:01.365583 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 22:47:01.365698 30325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 22:47:01.368183 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30196
I20260926 22:47:02.141476 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.141284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:46555)
0926 22:47:02.141398 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:02.141433 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260926 22:47:02.245316 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.245116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:43063)
0926 22:47:02.245229 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:02.245265 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260926 22:47:02.366154 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.365970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:37997)
0926 22:47:02.366080 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:02.366111 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260926 22:47:02.493362 30332 raft_consensus.cc:482] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:47:02.493543 30332 raft_consensus.cc:504] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:02.493887 30332 leader_election.cc:288] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823), 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031)
Sep 26 22:47:02 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462821, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.195@KRBTEST.COM
Sep 26 22:47:02 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462821, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:47:02.505613 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.494338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:33457 (local address 127.14.184.195:33823)
0926 22:47:02.494503 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:02.494513 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:02.494545 (+ 32us) server_negotiation.cc:407] Connection header received
0926 22:47:02.495178 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:02.495182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:02.495232 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:02.495373 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:02.495812 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.497086 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.498511 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.499165 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.499282 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:02.501766 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:02.501770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:02.501771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:02.501782 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:02.503845 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504364 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:02.504368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:02.504368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:02.504482 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504752 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:02.504753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:02.504754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:02.504993 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:02.505067 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:02.505372 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 22:47:02.505500 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":56,"threads_started":1}
I20260926 22:47:02.505605 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.494031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:44031 (local address 127.14.184.194:55753)
0926 22:47:02.494086 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:02.494115 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:02.494321 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:02.494769 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:02.494774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:02.495175 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:02.495468 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:02.495476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.497114 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.497115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:02.498417 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:02.498420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.499265 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.499266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:02.499401 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:02.499983 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:02.499994 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:02.502115 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:02.504089 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:02.504090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:02.504337 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:02.504608 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504608 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:02.504609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:02.504688 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:02.505104 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:02.505105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:02.505211 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 22:47:02.505320 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 22:47:02.505513 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":57}
I20260926 22:47:02.505731 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.494032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33823 (local address 127.14.184.194:33457)
0926 22:47:02.494162 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:02.494171 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:02.494319 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:02.495360 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:02.495361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:02.495608 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:02.495685 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:02.495687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.497163 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.497164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:02.498417 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:02.498420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.499266 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.499266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:02.499401 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:02.500011 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:02.500015 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:02.501656 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:02.503930 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:02.503937 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:02.503939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:02.504286 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:02.504561 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:02.504563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:02.504686 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:02.505074 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:02.505081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:02.505211 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 22:47:02.505320 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 22:47:02.505594 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":208,"thread_start_us":52,"threads_started":1}
I20260926 22:47:02.505640 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.494146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:55753 (local address 127.14.184.193:44031)
0926 22:47:02.494298 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:02.494304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:02.494319 (+ 15us) server_negotiation.cc:407] Connection header received
0926 22:47:02.494547 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:02.494549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:02.494594 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:02.494724 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:02.495590 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.497017 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.498511 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:02.499165 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:02.499282 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:02.502248 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:02.502252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:02.502253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:02.502264 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:02.504013 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504425 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:02.504428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:02.504428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:02.504531 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:02.504773 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:02.504774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:02.504774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:02.504996 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:02.505094 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:02.505375 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 22:47:02.505536 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
W20260926 22:47:02.506407 30078 leader_election.cc:341] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823): Not found: Tablet not found: f4c692b237dc4d47978530898fb43769
I20260926 22:47:02.506628 30010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c692b237dc4d47978530898fb43769" candidate_uuid: "26ca139c7b3b4e849ed7182136a752a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5552630dca0648349d5f7069f706bd58" is_pre_election: true
I20260926 22:47:02.506801 30010 raft_consensus.cc:2454] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26ca139c7b3b4e849ed7182136a752a0 in term 1.
I20260926 22:47:02.507081 30079 leader_election.cc:302] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26ca139c7b3b4e849ed7182136a752a0, 5552630dca0648349d5f7069f706bd58; no voters: 9c1d6efe13704c0e94719f67aec74e91
I20260926 22:47:02.507210 30332 raft_consensus.cc:2781] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 22:47:02.507273 30332 raft_consensus.cc:482] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:47:02.507296 30332 raft_consensus.cc:3037] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:47:02.508345 30332 raft_consensus.cc:504] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:02.508487 30332 leader_election.cc:288] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 2 election: Requested vote from peers 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823), 5552630dca0648349d5f7069f706bd58 (127.14.184.193:44031)
I20260926 22:47:02.508636 30010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c692b237dc4d47978530898fb43769" candidate_uuid: "26ca139c7b3b4e849ed7182136a752a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5552630dca0648349d5f7069f706bd58"
I20260926 22:47:02.508740 30010 raft_consensus.cc:3037] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 1 FOLLOWER]: Advancing to term 2
W20260926 22:47:02.508865 30078 leader_election.cc:341] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823): Not found: Tablet not found: f4c692b237dc4d47978530898fb43769
I20260926 22:47:02.509785 30010 raft_consensus.cc:2454] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26ca139c7b3b4e849ed7182136a752a0 in term 2.
I20260926 22:47:02.510007 30079 leader_election.cc:302] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26ca139c7b3b4e849ed7182136a752a0, 5552630dca0648349d5f7069f706bd58; no voters: 9c1d6efe13704c0e94719f67aec74e91
I20260926 22:47:02.510125 30332 raft_consensus.cc:2781] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:47:02.510253 30332 raft_consensus.cc:686] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [term 2 LEADER]: Becoming Leader. State: Replica: 26ca139c7b3b4e849ed7182136a752a0, State: Running, Role: LEADER
I20260926 22:47:02.510373 30332 consensus_queue.cc:237] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:02.510489 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:02.510406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:52181)
0926 22:47:02.510441 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:02.510471 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 22:47:03.058991 30078 consensus_peers.cc:487] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 -> Peer 9c1d6efe13704c0e94719f67aec74e91 (127.14.184.195:33823): Couldn't send request to peer 9c1d6efe13704c0e94719f67aec74e91. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4c692b237dc4d47978530898fb43769. This is attempt 1: this message will repeat every 5th retry.
I20260926 22:47:03.071934 30010 raft_consensus.cc:1261] T f4c692b237dc4d47978530898fb43769 P 5552630dca0648349d5f7069f706bd58 [term 2 FOLLOWER]: Refusing update from remote peer 26ca139c7b3b4e849ed7182136a752a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260926 22:47:03.072335 30338 consensus_queue.cc:1035] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 22:47:03.073369 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.073189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:46491)
0926 22:47:03.073283 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.073331 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":46,"threads_started":1}
I20260926 22:47:03.074287 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.074207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:50395)
0926 22:47:03.074234 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.074268 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260926 22:47:03.074309 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.074102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:48491)
0926 22:47:03.074217 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.074254 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":47,"threads_started":1}
I20260926 22:47:03.366801 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.366572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:34129)
0926 22:47:03.366735 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.366764 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
I20260926 22:47:03.485011 30347 ts_tablet_manager.cc:927] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Initiating tablet copy from peer 26ca139c7b3b4e849ed7182136a752a0 (127.14.184.194:33867)
I20260926 22:47:03.485221 30347 tablet_copy_client.cc:316] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: tablet copy: Beginning tablet copy session from remote peer at address 127.14.184.194:33867
Sep 26 22:47:03 dist-test-slave-3986 krb5kdc[28756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462821, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.194@KRBTEST.COM
I20260926 22:47:03.493185 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.485566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:39927 (local address 127.14.184.194:33867)
0926 22:47:03.485730 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:03.485734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:03.485742 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:47:03.485764 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:03.485766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:03.485778 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:03.485852 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:03.486338 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:03.487117 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:03.487981 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:03.488288 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:03.488368 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:03.490452 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:03.490459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:03.490459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:03.490467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:03.491934 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:03.492325 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:03.492329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:03.492329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:03.492392 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:03.492590 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:03.492591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:03.492591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:03.492734 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:03.492825 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:03.492997 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 22:47:03.493078 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260926 22:47:03.493199 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.485451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:33867 (local address 127.14.184.195:39927)
0926 22:47:03.485503 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.485523 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:03.485587 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:03.485873 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:03.485880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:03.486183 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:03.486241 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:03.486243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:03.487226 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:03.487227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:03.487881 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:03.487883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:03.488365 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:03.488366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:03.488459 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:03.488902 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:03.488906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:03.490341 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:03.492034 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:03.492040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:03.492042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:03.492229 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:03.492462 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:03.492463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:03.492463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:03.492524 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:03.492805 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:03.492810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:03.492862 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 22:47:03.492963 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:47:03.493078 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:47:03.493696 30165 tablet_copy_service.cc:140] P 26ca139c7b3b4e849ed7182136a752a0: Received BeginTabletCopySession request for tablet f4c692b237dc4d47978530898fb43769 from peer 9c1d6efe13704c0e94719f67aec74e91 ({username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:39927)
I20260926 22:47:03.493754 30165 tablet_copy_service.cc:161] P 26ca139c7b3b4e849ed7182136a752a0: Beginning new tablet copy session on tablet f4c692b237dc4d47978530898fb43769 from peer 9c1d6efe13704c0e94719f67aec74e91 at {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:39927: session id = 9c1d6efe13704c0e94719f67aec74e91-f4c692b237dc4d47978530898fb43769
I20260926 22:47:03.494181 30165 tablet_copy_source_session.cc:205] T f4c692b237dc4d47978530898fb43769 P 26ca139c7b3b4e849ed7182136a752a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 22:47:03.495134 30347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c692b237dc4d47978530898fb43769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:03.497664 30347 tablet_copy_client.cc:791] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: tablet copy: Starting download of 0 data blocks...
I20260926 22:47:03.497850 30347 tablet_copy_client.cc:655] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: tablet copy: Starting download of 1 WAL segments...
I20260926 22:47:03.498745 30347 tablet_copy_client.cc:531] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 22:47:03.499451 30347 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Bootstrap starting.
I20260926 22:47:03.500910 30347 log.cc:824] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:03.503417 30347 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:03.503638 30347 tablet_bootstrap.cc:492] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Bootstrap complete.
I20260926 22:47:03.503791 30347 ts_tablet_manager.cc:1397] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260926 22:47:03.504766 30347 raft_consensus.cc:348] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:03.504930 30347 raft_consensus.cc:729] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c1d6efe13704c0e94719f67aec74e91, State: Initialized, Role: FOLLOWER
I20260926 22:47:03.505064 30347 consensus_queue.cc:260] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26ca139c7b3b4e849ed7182136a752a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 33867 } } peers { permanent_uuid: "9c1d6efe13704c0e94719f67aec74e91" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33823 } } peers { permanent_uuid: "5552630dca0648349d5f7069f706bd58" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 44031 } }
I20260926 22:47:03.505303 30347 ts_tablet_manager.cc:1428] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20260926 22:47:03.505580 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.505477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:50973)
0926 22:47:03.505523 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.505550 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 22:47:03.505671 30165 tablet_copy_service.cc:342] P 26ca139c7b3b4e849ed7182136a752a0: Request end of tablet copy session 9c1d6efe13704c0e94719f67aec74e91-f4c692b237dc4d47978530898fb43769 received from {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:39927
I20260926 22:47:03.505738 30165 tablet_copy_service.cc:433] P 26ca139c7b3b4e849ed7182136a752a0: ending tablet copy session 9c1d6efe13704c0e94719f67aec74e91-f4c692b237dc4d47978530898fb43769 on tablet f4c692b237dc4d47978530898fb43769 with peer 9c1d6efe13704c0e94719f67aec74e91
I20260926 22:47:03.505870 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.505791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:50765)
0926 22:47:03.505823 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.505848 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:47:03.978376 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.978219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:56301)
0926 22:47:03.978299 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.978335 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":32,"threads_started":1}
I20260926 22:47:03.978338 30279 raft_consensus.cc:1203] T f4c692b237dc4d47978530898fb43769 P 9c1d6efe13704c0e94719f67aec74e91 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260926 22:47:03.978830 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:03.978713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:37359)
0926 22:47:03.978773 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:03.978804 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 22:47:04.075006 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:04.074775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:47995)
0926 22:47:04.074914 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:04.074947 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260926 22:47:04.979173 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:04.978907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:52379)
0926 22:47:04.979061 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:04.979125 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
I20260926 22:47:04.979480 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:04.979263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:46269)
0926 22:47:04.979412 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:04.979442 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
I20260926 22:47:05.075680 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:05.075459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:59417)
0926 22:47:05.075594 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:05.075633 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260926 22:47:05.979875 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:05.979603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:33677)
0926 22:47:05.979781 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:05.979830 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":57,"threads_started":1}
I20260926 22:47:05.980110 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:05.979908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:58723)
0926 22:47:05.980039 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:05.980079 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260926 22:47:06.076432 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:06.076170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:59495)
0926 22:47:06.076331 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:06.076371 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260926 22:47:06.980679 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:06.980412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:43443)
0926 22:47:06.980569 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:06.980615 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260926 22:47:06.980868 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:06.980631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:59409)
0926 22:47:06.980776 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:06.980819 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260926 22:47:06.981137 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:06.981023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:52683)
0926 22:47:06.981066 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:06.981094 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:47:07.077347 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:07.077067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:60077)
0926 22:47:07.077264 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:07.077301 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":59,"threads_started":1}
I20260926 22:47:07.981683 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:07.981433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:56067)
0926 22:47:07.981590 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:07.981627 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260926 22:47:07.981725 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:07.981542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:46845)
0926 22:47:07.981656 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:07.981686 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260926 22:47:08.077976 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:08.077778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:49889)
0926 22:47:08.077893 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:08.077928 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260926 22:47:08.982462 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:08.982248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:35875)
0926 22:47:08.982376 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:08.982410 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260926 22:47:08.982473 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:08.982248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:49259)
0926 22:47:08.982407 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:08.982436 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
I20260926 22:47:09.078836 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:09.078439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:51275)
0926 22:47:09.078745 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:09.078791 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":76,"threads_started":1}
I20260926 22:47:09.983165 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:09.982942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:43009)
0926 22:47:09.983086 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:09.983121 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260926 22:47:09.983287 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:09.983050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:58105)
0926 22:47:09.983175 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:09.983236 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260926 22:47:10.079654 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:10.079407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:36195)
0926 22:47:10.079568 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:10.079607 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260926 22:47:10.983840 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:10.983622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:46323)
0926 22:47:10.983768 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:10.983797 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260926 22:47:10.983909 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:10.983722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:58529)
0926 22:47:10.983832 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:10.983870 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260926 22:47:11.080261 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:11.080067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:32963)
0926 22:47:11.080183 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:11.080216 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260926 22:47:11.984489 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:11.984289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:39885)
0926 22:47:11.984419 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:11.984448 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260926 22:47:11.984675 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:11.984463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:43397)
0926 22:47:11.984593 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:11.984630 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260926 22:47:12.081007 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:12.080788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:33811)
0926 22:47:12.080918 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:12.080955 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260926 22:47:12.985251 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:12.985053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:60625)
0926 22:47:12.985172 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:12.985206 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260926 22:47:12.985291 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:12.985096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:42651)
0926 22:47:12.985224 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:12.985253 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260926 22:47:13.081806 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:13.081589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:34321)
0926 22:47:13.081728 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:13.081763 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260926 22:47:13.985982 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:13.985759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:43853)
0926 22:47:13.985901 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:13.985937 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260926 22:47:13.986023 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:13.985788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:34723)
0926 22:47:13.985926 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:13.985976 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260926 22:47:14.082480 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:14.082254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:50495)
0926 22:47:14.082399 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:14.082435 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260926 22:47:14.986737 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:14.986472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:60465)
0926 22:47:14.986602 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:14.986635 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260926 22:47:14.986748 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:14.986490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:57513)
0926 22:47:14.986631 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:14.986662 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260926 22:47:15.083251 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:15.082976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:55987)
0926 22:47:15.083164 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:15.083205 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260926 22:47:15.987409 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:15.987170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.195:40987)
0926 22:47:15.987313 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:15.987354 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260926 22:47:15.987468 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:15.987244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.194:55669)
0926 22:47:15.987392 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:15.987427 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260926 22:47:16.083941 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:16.083733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:38729 (local address 127.14.184.193:34205)
0926 22:47:16.083864 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:16.083897 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260926 22:47:16.368940 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 29897
I20260926 22:47:16.373767 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30059
I20260926 22:47:16.378475 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30196
2026-09-26T22:47:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18393 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:47:16 dist-test-slave-3986 krb5kdc[30396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:47:16 dist-test-slave-3986 krb5kdc[30396](info): set up 2 sockets
Sep 26 22:47:16 dist-test-slave-3986 krb5kdc[30396](info): commencing operation
krb5kdc: starting...
W20260926 22:47:17.399101 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:47:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:47:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:47:17.517259 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43523
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:17.596537 30409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:17.596787 30409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:17.596824 30409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:17.598008 30409 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 22:47:17.598078 30409 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 22:47:17.598099 30409 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 22:47:17.598119 30409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:47:17.598147 30409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:17.598173 30409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:17.598203 30409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:47:17.598227 30409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:47:17.599768 30409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:17.600044 30409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:17.600273 30409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:47:17.605307 30409 server_base.cc:1034] running on GCE node
W20260926 22:47:17.605376 30417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:17.605350 30415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:17.605386 30414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:17.605898 30409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:17.606210 30409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:17.607367 30409 hybrid_clock.cc:648] HybridClock initialized: now 1790462837607363 us; error 38 us; skew 500 ppm
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:17.609769 30409 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:47:17.610160 30409 webserver.cc:460] Webserver started at http://127.14.184.254:34373/ using document root <none> and password file <none>
I20260926 22:47:17.610358 30409 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:17.610416 30409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:17.610544 30409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:47:17.611557 30409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "70be308833824315bea28f9439dc6391"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "ecb344d0b60de69d756c6d9a00a06e9a"
server_key_iv: "8616cb97e114e073579b5ba510cdc395"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.611909 30409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "70be308833824315bea28f9439dc6391"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "ecb344d0b60de69d756c6d9a00a06e9a"
server_key_iv: "8616cb97e114e073579b5ba510cdc395"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.613148 30409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:47:17.613821 30424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.613986 30409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:17.614081 30409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "70be308833824315bea28f9439dc6391"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "ecb344d0b60de69d756c6d9a00a06e9a"
server_key_iv: "8616cb97e114e073579b5ba510cdc395"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.614161 30409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:17.622417 30409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:17.624423 30409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:17.624579 30409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:17.628399 30409 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43523
I20260926 22:47:17.628429 30476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43523 every 8 connection(s)
I20260926 22:47:17.628755 30409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:47:17.629595 30477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:17.631886 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30409
I20260926 22:47:17.631953 30477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap starting.
I20260926 22:47:17.631989 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:47:17.632174 15075 external_mini_cluster.cc:1428] Setting key c6996efa9c27ccb75f4647b02a8a44b0
I20260926 22:47:17.632650 30477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:17.633041 30477 log.cc:824] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:17.633833 30477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: No bootstrap required, opened a new log
I20260926 22:47:17.636099 30477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:17.636261 30477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:17.636298 30477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70be308833824315bea28f9439dc6391, State: Initialized, Role: FOLLOWER
I20260926 22:47:17.636425 30477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:17.636519 30477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:47:17.636570 30477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:47:17.636619 30477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:17.637221 30477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:17.637336 30477 leader_election.cc:302] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70be308833824315bea28f9439dc6391; no voters:
I20260926 22:47:17.637501 30477 leader_election.cc:288] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:47:17.637671 30482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:47:17.637792 30477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:47:17.637848 30482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 LEADER]: Becoming Leader. State: Replica: 70be308833824315bea28f9439dc6391, State: Running, Role: LEADER
I20260926 22:47:17.638113 30482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:17.638504 30483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70be308833824315bea28f9439dc6391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } } }
I20260926 22:47:17.638504 30484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70be308833824315bea28f9439dc6391. Latest consensus state: current_term: 1 leader_uuid: "70be308833824315bea28f9439dc6391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } } }
I20260926 22:47:17.638621 30483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:17.638625 30484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:17.638969 30491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:47:17.639460 30491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:17.641124 30491 catalog_manager.cc:1347] Generated new cluster ID: 4682a95dfc2a456fbf78f3c0a16afbbd
I20260926 22:47:17.641178 30491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:47:17.645181 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:17.632731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.14.184.254:43523)
0926 22:47:17.632896 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:17.632903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:17.632927 (+ 24us) server_negotiation.cc:407] Connection header received
0926 22:47:17.633442 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:17.633450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:17.633513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:17.633628 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:17.634370 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.635338 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.636471 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.636973 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.637052 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:17.640161 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:17.640165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:17.640165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:17.640177 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:17.643590 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:17.644010 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:17.644012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:17.644013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:17.644110 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:17.644323 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:17.644323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:17.644324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:17.644590 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:17.644719 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:17.644878 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 22:47:17.645001 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260926 22:47:17.656587 30491 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:47:17.657330 30491 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:47:17.663085 30491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Generated new TSK 0
I20260926 22:47:17.663267 30491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:47:17.693349 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43523
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:17.792023 30503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:17.792232 30503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:17.792258 30503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:17.793426 30503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:17.793468 30503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:17.793496 30503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:47:17.794982 30503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:17.795212 30503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:17.795456 30503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:17.800845 30511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:17.800853 30508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:17.800910 30503 server_base.cc:1034] running on GCE node
W20260926 22:47:17.800856 30509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:17.801255 30503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:17.801507 30503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:17.802699 30503 hybrid_clock.cc:648] HybridClock initialized: now 1790462837802656 us; error 32 us; skew 500 ppm
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:17.805158 30503 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:47:17.805544 30503 webserver.cc:460] Webserver started at http://127.14.184.193:44451/ using document root <none> and password file <none>
I20260926 22:47:17.805717 30503 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:17.805763 30503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:17.805851 30503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:47:17.806885 30503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "99e89883330b4024ab6d4a2c970bc2f7"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "7fdb22a4da658fd095e0f3cf99c60928"
server_key_iv: "36673c1d46d7e15986b49b6ed35b748b"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.807178 30503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "99e89883330b4024ab6d4a2c970bc2f7"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "7fdb22a4da658fd095e0f3cf99c60928"
server_key_iv: "36673c1d46d7e15986b49b6ed35b748b"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.808487 30503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:47:17.809216 30518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.809427 30503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:17.809513 30503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "99e89883330b4024ab6d4a2c970bc2f7"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "7fdb22a4da658fd095e0f3cf99c60928"
server_key_iv: "36673c1d46d7e15986b49b6ed35b748b"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.809567 30503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:17.823451 30503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:17.825299 30503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:17.825412 30503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:17.825590 30503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:17.825963 30503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:17.826000 30503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.826045 30503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:17.826071 30503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.831516 30503 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:45677
I20260926 22:47:17.831554 30631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:45677 every 8 connection(s)
I20260926 22:47:17.831908 30503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:17.838474 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30503
I20260926 22:47:17.838596 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:47:17.839097 15075 external_mini_cluster.cc:1428] Setting key 55f1088ef04fa5fabfcad9e5b3ec2302
I20260926 22:47:17.841584 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:17.832656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:36147)
0926 22:47:17.832900 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:17.832936 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:17.833401 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:17.833625 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:17.833629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:17.834190 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:17.834392 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:17.834397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.835167 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.835168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:17.835972 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:17.835974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.836344 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.836345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:17.836440 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:17.836849 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:17.836857 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:17.838244 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:17.840190 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:17.840196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:17.840198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:17.840413 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:17.840640 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:17.840641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:17.840642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:17.840744 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:17.841081 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:17.841089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:17.841236 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 22:47:17.841325 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 22:47:17.841436 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260926 22:47:17.841585 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:17.832693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:36147 (local address 127.14.184.254:43523)
0926 22:47:17.832754 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:17.832756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:17.833016 (+ 260us) server_negotiation.cc:407] Connection header received
0926 22:47:17.833487 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:17.833488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:17.833497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:17.833549 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:17.834483 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.835087 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.836052 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.836269 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.836325 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:17.838421 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:17.838423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:17.838423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:17.838430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:17.840079 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:17.840507 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:17.840509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:17.840509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:17.840572 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:17.840827 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:17.840828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:17.840828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:17.840986 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:17.841064 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:17.841369 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 22:47:17.841465 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:47:17.842094 30632 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43523
I20260926 22:47:17.842203 30632 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:17.842378 30632 heartbeater.cc:511] Master 127.14.184.254:43523 requested a full tablet report, sending...
I20260926 22:47:17.842934 30441 ts_manager.cc:194] Registered new tserver with Master: 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677)
I20260926 22:47:17.843525 30441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:36147
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:47:17.864816 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43523
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:17.946619 30637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:17.946825 30637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:17.946847 30637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:17.947919 30637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:17.947952 30637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:17.947978 30637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:47:17.949384 30637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:17.949548 30637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:17.949712 30637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:47:17.954481 30637 server_base.cc:1034] running on GCE node
W20260926 22:47:17.954432 30642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:17.954442 30643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:17.954442 30645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:17.954944 30637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:17.955168 30637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:17.956313 30637 hybrid_clock.cc:648] HybridClock initialized: now 1790462837956305 us; error 31 us; skew 500 ppm
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:17.958719 30637 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:47:17.959102 30637 webserver.cc:460] Webserver started at http://127.14.184.194:40911/ using document root <none> and password file <none>
I20260926 22:47:17.959309 30637 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:17.959352 30637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:17.959483 30637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:47:17.960421 30637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "9c77c86a5c323d29ec4017fc6f31487d"
server_key_iv: "343112204d2c18293c39401a4707e42d"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.960745 30637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "9c77c86a5c323d29ec4017fc6f31487d"
server_key_iv: "343112204d2c18293c39401a4707e42d"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.961952 30637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:47:17.962590 30652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.962780 30637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:17.962875 30637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "9c77c86a5c323d29ec4017fc6f31487d"
server_key_iv: "343112204d2c18293c39401a4707e42d"
server_key_version: "encryptionkey@0"
I20260926 22:47:17.962966 30637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:17.983762 30637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:17.985654 30637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:17.985818 30637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:17.986052 30637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:17.986362 30637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:17.986415 30637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.986450 30637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:17.986477 30637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:17.991952 30637 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:44035
I20260926 22:47:17.992003 30765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:44035 every 8 connection(s)
I20260926 22:47:17.992324 30637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:47:17 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462837, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:17.999583 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30637
I20260926 22:47:17.999698 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:47:17.999895 15075 external_mini_cluster.cc:1428] Setting key b65de24076181703c66a3dd6451b6257
I20260926 22:47:18.001075 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:17.993006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:50037)
0926 22:47:17.993175 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:17.993207 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:17.993641 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:17.993828 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:17.993831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:17.994347 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:17.994556 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:17.994560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.995221 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.995222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:17.996000 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:17.996001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.996322 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.996323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:17.996416 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:17.996851 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:17.996858 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:17.998108 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:17.999634 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:17.999657 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:17.999659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:17.999984 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:18.000247 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:18.000248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:18.000249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:18.000344 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:18.000606 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:18.000611 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:18.000729 (+ 118us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.000821 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.000929 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260926 22:47:18.001084 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:17.993031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:50037 (local address 127.14.184.254:43523)
0926 22:47:17.993103 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:17.993105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:17.993294 (+ 189us) server_negotiation.cc:407] Connection header received
0926 22:47:17.993716 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:17.993717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:17.993726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:17.993778 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:17.994640 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.995148 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.996070 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:17.996244 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:17.996303 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:17.998182 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:17.998183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:17.998184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:17.998190 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:17.999554 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:18.000047 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:18.000048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:18.000048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:18.000107 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:18.000408 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:18.000409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:18.000409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:18.000523 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:18.000603 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.000866 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.000945 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 22:47:18.001523 30766 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43523
I20260926 22:47:18.001617 30766 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:18.001802 30766 heartbeater.cc:511] Master 127.14.184.254:43523 requested a full tablet report, sending...
I20260926 22:47:18.002141 30441 ts_manager.cc:194] Registered new tserver with Master: 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035)
I20260926 22:47:18.002506 30441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:50037
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:47:18.023813 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43523
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:18.102424 30771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:18.102622 30771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:18.102646 30771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:18.103792 30771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:18.103833 30771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:18.103859 30771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:47:18.105176 30771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:18.105363 30771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:18.105522 30771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:18.110633 30777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:18.110641 30776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:18.110625 30779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:18.110730 30771 server_base.cc:1034] running on GCE node
I20260926 22:47:18.111058 30771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:18.111299 30771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:18.112437 30771 hybrid_clock.cc:648] HybridClock initialized: now 1790462838112405 us; error 47 us; skew 500 ppm
Sep 26 22:47:18 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:18.114948 30771 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:47:18.115381 30771 webserver.cc:460] Webserver started at http://127.14.184.195:45017/ using document root <none> and password file <none>
I20260926 22:47:18.115559 30771 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:18.115597 30771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:18.115682 30771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:47:18.116637 30771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "bb545787c354421198422a4336dad159"
format_stamp: "Formatted at 2026-09-26 22:47:18 on dist-test-slave-3986"
server_key: "adb1eb727fe57b4856e958ab320ecf91"
server_key_iv: "ba0a7023c686624a3398ebeaa77f8bf0"
server_key_version: "encryptionkey@0"
I20260926 22:47:18.116938 30771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "bb545787c354421198422a4336dad159"
format_stamp: "Formatted at 2026-09-26 22:47:18 on dist-test-slave-3986"
server_key: "adb1eb727fe57b4856e958ab320ecf91"
server_key_iv: "ba0a7023c686624a3398ebeaa77f8bf0"
server_key_version: "encryptionkey@0"
I20260926 22:47:18.118163 30771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:47:18.118901 30786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:18.119084 30771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:18.119153 30771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "bb545787c354421198422a4336dad159"
format_stamp: "Formatted at 2026-09-26 22:47:18 on dist-test-slave-3986"
server_key: "adb1eb727fe57b4856e958ab320ecf91"
server_key_iv: "ba0a7023c686624a3398ebeaa77f8bf0"
server_key_version: "encryptionkey@0"
I20260926 22:47:18.119203 30771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:18.128650 30771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:18.130295 30771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:18.130403 30771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:18.130564 30771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:18.130865 30771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:18.130898 30771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:18.130918 30771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:18.130930 30771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:18.136780 30771 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33955
I20260926 22:47:18.136839 30899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33955 every 8 connection(s)
I20260926 22:47:18.137120 30771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:47:18.138226 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30771
I20260926 22:47:18.138334 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:47:18.138572 15075 external_mini_cluster.cc:1428] Setting key 879bc15855cf51627cc372811824e5bb
Sep 26 22:47:18 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:18.146229 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.137834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:48599 (local address 127.14.184.254:43523)
0926 22:47:18.137891 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.137893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.138126 (+ 233us) server_negotiation.cc:407] Connection header received
0926 22:47:18.138881 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.138883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.138893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.138953 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:18.139934 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.140447 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.141437 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.141644 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.141736 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.143821 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:18.143823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:18.143823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:18.143829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:18.145029 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:18.145398 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:18.145399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:18.145400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:18.145464 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:18.145686 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:18.145687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:18.145687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:18.145789 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:18.145842 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.146096 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.146164 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 22:47:18.146308 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.137808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:48599)
0926 22:47:18.137999 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.138038 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.138753 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.139026 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.139030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.139594 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:18.139847 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.139854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.140550 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.140551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.141346 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.141348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.141747 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.141748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.141871 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.142338 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:18.142346 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:18.143731 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:18.145100 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:18.145105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:18.145107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:18.145322 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:18.145539 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:18.145539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:18.145540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:18.145631 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:18.145844 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:18.145848 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:18.145957 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.146038 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.146153 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":132,"threads_started":1}
I20260926 22:47:18.146813 30900 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43523
I20260926 22:47:18.146908 30900 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:18.147049 30900 heartbeater.cc:511] Master 127.14.184.254:43523 requested a full tablet report, sending...
I20260926 22:47:18.147428 30441 ts_manager.cc:194] Registered new tserver with Master: bb545787c354421198422a4336dad159 (127.14.184.195:33955)
I20260926 22:47:18.147815 30441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:48599
I20260926 22:47:18.150161 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:47:18.156844 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.151488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.14.184.254:43523)
0926 22:47:18.151528 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.151530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.151625 (+ 95us) server_negotiation.cc:407] Connection header received
0926 22:47:18.151676 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.151676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.151684 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.151731 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:18.152234 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.152700 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.153496 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.153672 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.153726 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.154853 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:18.154854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:18.154854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:18.154860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:18.155959 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:18.156248 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:18.156249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:18.156258 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:18.156308 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:18.156452 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:18.156452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:18.156452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:18.156573 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:18.156645 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.156723 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.156783 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:47:18.158264 30441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:47:18.159868 30441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:47:18.165972 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.161796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:44035 (local address 127.0.0.1:53756)
0926 22:47:18.161939 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.161983 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.162049 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.162336 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.162339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.162342 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.162408 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.162410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.163753 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.163754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.164785 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.164788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.165667 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.165668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.165760 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.165774 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.165850 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.165899 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260926 22:47:18.166057 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.161912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.14.184.194:44035)
0926 22:47:18.162114 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.162119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.162130 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:47:18.162193 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.162195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.162213 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.162330 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.162503 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.163636 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.164851 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.165580 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.165660 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.165714 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.165919 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.165968 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
I20260926 22:47:18.166368 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.162345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.14.184.193:45677)
0926 22:47:18.162533 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.162538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.162548 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:47:18.162585 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.162587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.162602 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.162700 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.162916 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.163945 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.165063 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.165786 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.165950 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.166019 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.166225 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.166270 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260926 22:47:18.166289 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.162147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.0.0.1:46608)
0926 22:47:18.162408 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.162417 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.162453 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.162713 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.162714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.162716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.162842 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.162844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.164043 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.164044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.164971 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.164973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.166059 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.166060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.166129 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.166134 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.166186 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.166227 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":37,"threads_started":1}
I20260926 22:47:18.167371 30700 tablet_service.cc:1467] Processing CreateTablet for tablet a250035f950547f6ae963fd94f1400fc (DEFAULT_TABLE table=test-table [id=1bb771c7087248acbe08f6a161dc105a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:18.167649 30700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a250035f950547f6ae963fd94f1400fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:18.167518 30566 tablet_service.cc:1467] Processing CreateTablet for tablet a250035f950547f6ae963fd94f1400fc (DEFAULT_TABLE table=test-table [id=1bb771c7087248acbe08f6a161dc105a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:18.167866 30566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a250035f950547f6ae963fd94f1400fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:18.168788 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.162100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.14.184.195:33955)
0926 22:47:18.162283 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.162287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.164871 (+ 2584us) server_negotiation.cc:407] Connection header received
0926 22:47:18.164921 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.164923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.164936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.165015 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.165176 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.166192 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.167690 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.168424 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.168504 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.168553 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.168667 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.168710 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260926 22:47:18.168835 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.162009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33955 (local address 127.0.0.1:44434)
0926 22:47:18.162254 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.162264 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.162332 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.165024 (+ 2692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.165026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.165028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.165066 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.165067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.166304 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.166306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.167577 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.167580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.168525 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.168526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.168589 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.168593 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.168657 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.168765 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":48,"threads_started":1}
I20260926 22:47:18.169495 30834 tablet_service.cc:1467] Processing CreateTablet for tablet a250035f950547f6ae963fd94f1400fc (DEFAULT_TABLE table=test-table [id=1bb771c7087248acbe08f6a161dc105a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:18.169806 30834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a250035f950547f6ae963fd94f1400fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:18.170611 30914 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap starting.
I20260926 22:47:18.171284 30915 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Bootstrap starting.
I20260926 22:47:18.171371 30914 tablet_bootstrap.cc:654] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:18.171967 30915 tablet_bootstrap.cc:654] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:18.171949 30914 log.cc:824] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:18.172477 30915 log.cc:824] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:18.172758 30914 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: No bootstrap required, opened a new log
I20260926 22:47:18.172832 30914 ts_tablet_manager.cc:1397] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:18.172876 30918 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Bootstrap starting.
I20260926 22:47:18.173398 30915 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: No bootstrap required, opened a new log
I20260926 22:47:18.173468 30915 ts_tablet_manager.cc:1397] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:47:18.174821 30914 raft_consensus.cc:348] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.175001 30914 raft_consensus.cc:374] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:18.175037 30914 raft_consensus.cc:729] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e89883330b4024ab6d4a2c970bc2f7, State: Initialized, Role: FOLLOWER
I20260926 22:47:18.175212 30918 tablet_bootstrap.cc:654] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:18.175179 30914 consensus_queue.cc:260] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.175467 30914 ts_tablet_manager.cc:1428] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:47:18.175465 30915 raft_consensus.cc:348] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.175603 30632 heartbeater.cc:503] Master 127.14.184.254:43523 was elected leader, sending a full tablet report...
I20260926 22:47:18.175633 30915 raft_consensus.cc:374] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:18.175675 30915 raft_consensus.cc:729] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f047af5bc242d4a715aaf2bb0d80a0, State: Initialized, Role: FOLLOWER
I20260926 22:47:18.175735 30918 log.cc:824] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:18.175808 30915 consensus_queue.cc:260] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.176098 30915 ts_tablet_manager.cc:1428] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:47:18.176137 30766 heartbeater.cc:503] Master 127.14.184.254:43523 was elected leader, sending a full tablet report...
I20260926 22:47:18.176867 30918 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: No bootstrap required, opened a new log
I20260926 22:47:18.176937 30918 ts_tablet_manager.cc:1397] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 22:47:18.178290 30918 raft_consensus.cc:348] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.178445 30918 raft_consensus.cc:374] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:18.178489 30918 raft_consensus.cc:729] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb545787c354421198422a4336dad159, State: Initialized, Role: FOLLOWER
I20260926 22:47:18.178591 30918 consensus_queue.cc:260] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.178926 30918 ts_tablet_manager.cc:1428] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:47:18.178941 30900 heartbeater.cc:503] Master 127.14.184.254:43523 was elected leader, sending a full tablet report...
W20260926 22:47:18.242887 30767 tablet.cc:2369] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:47:18.286633 30921 raft_consensus.cc:482] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:47:18.286774 30921 raft_consensus.cc:504] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.287086 30921 leader_election.cc:288] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb545787c354421198422a4336dad159 (127.14.184.195:33955), 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677)
I20260926 22:47:18.290091 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.287477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:53117 (local address 127.14.184.195:33955)
0926 22:47:18.287527 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.287534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.287541 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:47:18.287564 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.287565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.287572 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.287638 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.287818 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.288292 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.289338 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.289720 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.289810 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.289820 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.289981 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.290018 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:47:18.290091 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.287279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33955 (local address 127.14.184.194:53117)
0926 22:47:18.287367 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.287386 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.287444 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.287645 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.287646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.287648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.287727 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.287729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.288381 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.288382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.289242 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.289244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.289813 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.289814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.289893 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.289896 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.289971 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.290015 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 22:47:18.290172 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.287481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:32785 (local address 127.14.184.193:45677)
0926 22:47:18.287528 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.287530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.287628 (+ 98us) server_negotiation.cc:407] Connection header received
0926 22:47:18.287695 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.287696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.287705 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.287783 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.287936 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.288401 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.289399 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.289804 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.289889 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.289897 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.290050 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.290088 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:47:18.290254 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.287386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.14.184.194:32785)
0926 22:47:18.287545 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.287555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.287637 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.287784 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.287785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.287786 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.287865 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.287866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.288488 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.288489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.289312 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.289314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.289902 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.289903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.289970 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.289973 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.290043 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.290082 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260926 22:47:18.290335 30922 raft_consensus.cc:482] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:47:18.290445 30922 raft_consensus.cc:504] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.290501 30854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a250035f950547f6ae963fd94f1400fc" candidate_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb545787c354421198422a4336dad159" is_pre_election: true
I20260926 22:47:18.290627 30854 raft_consensus.cc:2454] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f047af5bc242d4a715aaf2bb0d80a0 in term 0.
I20260926 22:47:18.290808 30922 leader_election.cc:288] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035), 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677)
I20260926 22:47:18.290977 30655 leader_election.cc:302] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f047af5bc242d4a715aaf2bb0d80a0, bb545787c354421198422a4336dad159; no voters:
I20260926 22:47:18.291164 30921 raft_consensus.cc:2781] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:47:18.291260 30921 raft_consensus.cc:482] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:47:18.291326 30921 raft_consensus.cc:3037] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:18.291352 30586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a250035f950547f6ae963fd94f1400fc" candidate_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e89883330b4024ab6d4a2c970bc2f7" is_pre_election: true
I20260926 22:47:18.291510 30586 raft_consensus.cc:2454] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f047af5bc242d4a715aaf2bb0d80a0 in term 0.
I20260926 22:47:18.293431 30921 raft_consensus.cc:504] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.293735 30921 leader_election.cc:288] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [CANDIDATE]: Term 1 election: Requested vote from peers bb545787c354421198422a4336dad159 (127.14.184.195:33955), 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677)
I20260926 22:47:18.293716 30854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a250035f950547f6ae963fd94f1400fc" candidate_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb545787c354421198422a4336dad159"
I20260926 22:47:18.293792 30854 raft_consensus.cc:3037] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:18.294147 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.290953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:44035 (local address 127.14.184.195:49889)
0926 22:47:18.291006 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.291018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.291055 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.291217 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.291218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.291220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.291299 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.291300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.292080 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.292080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.293164 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.293166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.293876 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.293877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.293967 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.293970 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.294044 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.294084 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:47:18.294143 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.291020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:49889 (local address 127.14.184.194:44035)
0926 22:47:18.291058 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.291060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.291068 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:47:18.291111 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.291112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.291121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.291173 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.291374 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.292006 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.293259 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.293808 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.293868 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.293876 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.294052 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.294084 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:47:18.294412 30720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a250035f950547f6ae963fd94f1400fc" candidate_uuid: "bb545787c354421198422a4336dad159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" is_pre_election: true
I20260926 22:47:18.294492 30720 raft_consensus.cc:2379] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb545787c354421198422a4336dad159 in current term 1: Already voted for candidate 15f047af5bc242d4a715aaf2bb0d80a0 in this term.
I20260926 22:47:18.294643 30854 raft_consensus.cc:2454] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f047af5bc242d4a715aaf2bb0d80a0 in term 1.
I20260926 22:47:18.294816 30655 leader_election.cc:302] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f047af5bc242d4a715aaf2bb0d80a0, bb545787c354421198422a4336dad159; no voters:
I20260926 22:47:18.294837 30586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a250035f950547f6ae963fd94f1400fc" candidate_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e89883330b4024ab6d4a2c970bc2f7"
I20260926 22:47:18.294893 30921 raft_consensus.cc:2781] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:47:18.294906 30586 raft_consensus.cc:3037] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:18.295055 30921 raft_consensus.cc:686] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 LEADER]: Becoming Leader. State: Replica: 15f047af5bc242d4a715aaf2bb0d80a0, State: Running, Role: LEADER
I20260926 22:47:18.295110 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.291277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:34591 (local address 127.14.184.193:45677)
0926 22:47:18.291314 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.291315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.292028 (+ 713us) server_negotiation.cc:407] Connection header received
0926 22:47:18.292058 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.292059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.292069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.292156 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:18.292328 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.292981 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.294148 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.294736 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.294785 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.294793 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.295007 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.295055 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:47:18.295118 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.291069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.14.184.195:34591)
0926 22:47:18.291943 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.291953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:18.292004 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:18.292157 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:18.292158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:18.292160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:18.292237 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.292238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.293068 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.293069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.294046 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:18.294048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.294851 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.294851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:18.294922 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.294925 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:18.294982 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 22:47:18.295023 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":119,"threads_started":1}
I20260926 22:47:18.295145 30921 consensus_queue.cc:237] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:18.295446 30585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a250035f950547f6ae963fd94f1400fc" candidate_uuid: "bb545787c354421198422a4336dad159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99e89883330b4024ab6d4a2c970bc2f7" is_pre_election: true
I20260926 22:47:18.295781 30586 raft_consensus.cc:2454] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f047af5bc242d4a715aaf2bb0d80a0 in term 1.
I20260926 22:47:18.295864 30789 leader_election.cc:302] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb545787c354421198422a4336dad159; no voters: 15f047af5bc242d4a715aaf2bb0d80a0, 99e89883330b4024ab6d4a2c970bc2f7
I20260926 22:47:18.295974 30922 raft_consensus.cc:2726] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 22:47:18.296082 30441 catalog_manager.cc:5495] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260926 22:47:18.312966 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.310739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.14.184.194:44035)
0926 22:47:18.310790 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:18.310792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:18.310799 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:47:18.310868 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:18.310869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:18.310884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:18.310942 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:47:18.311042 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.311501 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.312292 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:18.312469 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:18.312522 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:18.312641 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:47:18.312735 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:47:18.312798 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:18.312830 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 22:47:18.312846 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:47:18.314618 30586 raft_consensus.cc:1261] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Refusing update from remote peer 15f047af5bc242d4a715aaf2bb0d80a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:47:18.314622 30854 raft_consensus.cc:1261] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Refusing update from remote peer 15f047af5bc242d4a715aaf2bb0d80a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:47:18.314988 30925 consensus_queue.cc:1035] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:47:18.315105 30921 consensus_queue.cc:1035] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:47:18.317402 30928 mvcc.cc:205] Tried to move back new op lower bound from 7333735785734258688 to 7333735785657511936. Current Snapshot: MvccSnapshot[applied={T|T < 7333735785734258688}]
I20260926 22:47:18.317929 30930 mvcc.cc:205] Tried to move back new op lower bound from 7333735785734258688 to 7333735785657511936. Current Snapshot: MvccSnapshot[applied={T|T < 7333735785734258688}]
I20260926 22:47:18.320010 30439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:47:18.320113 30439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:47:18.320870 30439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Sending DeleteTablet for 3 replicas of tablet a250035f950547f6ae963fd94f1400fc
I20260926 22:47:18.321200 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30503
I20260926 22:47:18.321301 30566 tablet_service.cc:1514] Processing DeleteTablet for tablet a250035f950547f6ae963fd94f1400fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:18 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:46608
I20260926 22:47:18.321725 30700 tablet_service.cc:1514] Processing DeleteTablet for tablet a250035f950547f6ae963fd94f1400fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:18 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:53756
I20260926 22:47:18.321758 30834 tablet_service.cc:1514] Processing DeleteTablet for tablet a250035f950547f6ae963fd94f1400fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:18 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:44434
I20260926 22:47:18.323920 30938 tablet_replica.cc:330] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: stopping tablet replica
I20260926 22:47:18.324033 30938 raft_consensus.cc:2229] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:47:18.324160 30938 raft_consensus.cc:2258] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:18.324611 30938 ts_tablet_manager.cc:1905] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:47:18.326623 30938 ts_tablet_manager.cc:1918] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:47:18.326742 30938 log.cc:1196] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/a250035f950547f6ae963fd94f1400fc
I20260926 22:47:18.327030 30938 ts_tablet_manager.cc:1939] T a250035f950547f6ae963fd94f1400fc P 15f047af5bc242d4a715aaf2bb0d80a0: Deleting consensus metadata
I20260926 22:47:18.327425 30939 tablet_replica.cc:330] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: stopping tablet replica
I20260926 22:47:18.327472 30427 catalog_manager.cc:4841] TS 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035): tablet a250035f950547f6ae963fd94f1400fc (table test-table [id=1bb771c7087248acbe08f6a161dc105a]) successfully deleted
I20260926 22:47:18.327535 30939 raft_consensus.cc:2229] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:18.327653 30939 raft_consensus.cc:2258] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 22:47:18.328055 30427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:45677 (error 108)
I20260926 22:47:18.328085 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30637
I20260926 22:47:18.329183 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:18.329060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.0.0.1:46624)
0926 22:47:18.329114 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:18.329157 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 22:47:18.329334 30427 catalog_manager.cc:4584] TS 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a250035f950547f6ae963fd94f1400fc: Network error: Client connection negotiation failed: client connection to 127.14.184.193:45677: connect: Connection refused (error 111)
I20260926 22:47:18.329569 30939 ts_tablet_manager.cc:1905] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:47:18.331521 30939 ts_tablet_manager.cc:1918] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:47:18.331593 30939 log.cc:1196] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/a250035f950547f6ae963fd94f1400fc
I20260926 22:47:18.332197 30939 ts_tablet_manager.cc:1939] T a250035f950547f6ae963fd94f1400fc P bb545787c354421198422a4336dad159: Deleting consensus metadata
I20260926 22:47:18.332582 30427 catalog_manager.cc:4841] TS bb545787c354421198422a4336dad159 (127.14.184.195:33955): tablet a250035f950547f6ae963fd94f1400fc (table test-table [id=1bb771c7087248acbe08f6a161dc105a]) successfully deleted
I20260926 22:47:18.333914 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30771
I20260926 22:47:18.338735 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30409
I20260926 22:47:38.343953 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43523
--webserver_interface=127.14.184.254
--webserver_port=34373
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:38.423123 30941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:38.423338 30941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:38.423372 30941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:38.424453 30941 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 22:47:38.424513 30941 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 22:47:38.424535 30941 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 22:47:38.424561 30941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:47:38.424585 30941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:38.424616 30941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:38.424640 30941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:47:38.424664 30941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:47:38.426098 30941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=34373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:38.426371 30941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:38.426584 30941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:38.434568 30947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.434592 30948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.434568 30950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:38.434902 30941 server_base.cc:1034] running on GCE node
I20260926 22:47:38.435153 30941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:38.435484 30941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:38.436652 30941 hybrid_clock.cc:648] HybridClock initialized: now 1790462858436638 us; error 44 us; skew 500 ppm
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:38.439254 30941 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:47:38.439731 30941 webserver.cc:460] Webserver started at http://127.14.184.254:34373/ using document root <none> and password file <none>
I20260926 22:47:38.439958 30941 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:38.440037 30941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:38.441356 30941 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 22:47:38.442015 30957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.442183 30941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.442289 30941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "70be308833824315bea28f9439dc6391"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "ecb344d0b60de69d756c6d9a00a06e9a"
server_key_iv: "8616cb97e114e073579b5ba510cdc395"
server_key_version: "encryptionkey@0"
I20260926 22:47:38.442678 30941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:38.463599 30941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:38.466590 30941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:38.466833 30941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:38.471108 30941 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43523
I20260926 22:47:38.471138 31009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43523 every 8 connection(s)
I20260926 22:47:38.471511 30941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:47:38.472901 31010 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:47:38.473567 31010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap starting.
I20260926 22:47:38.479043 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 30941
I20260926 22:47:38.479332 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:45677
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:43523
--webserver_port=44451
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:38.481812 31010 log.cc:824] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:38.483731 31010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:38.484025 31010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap complete.
I20260926 22:47:38.486243 31010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:38.486490 31010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70be308833824315bea28f9439dc6391, State: Initialized, Role: FOLLOWER
I20260926 22:47:38.486717 31010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:38.486819 31010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:47:38.486879 31010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:47:38.486927 31010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:47:38.487910 31010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:38.488027 31010 leader_election.cc:302] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70be308833824315bea28f9439dc6391; no voters:
I20260926 22:47:38.488222 31010 leader_election.cc:288] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 22:47:38.488273 31014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:47:38.488430 31014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 LEADER]: Becoming Leader. State: Replica: 70be308833824315bea28f9439dc6391, State: Running, Role: LEADER
I20260926 22:47:38.488521 31010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:47:38.488593 31014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:38.488906 31014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70be308833824315bea28f9439dc6391. Latest consensus state: current_term: 2 leader_uuid: "70be308833824315bea28f9439dc6391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } } }
I20260926 22:47:38.488907 31015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "70be308833824315bea28f9439dc6391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } } }
I20260926 22:47:38.489173 31015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:38.489589 31023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:47:38.490329 31023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:47:38.490428 31023 catalog_manager.cc:1259] Loaded cluster ID: 4682a95dfc2a456fbf78f3c0a16afbbd
I20260926 22:47:38.490460 31023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:47:38.490792 31014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:38.491573 31023 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:47:38.491756 31023 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Loaded TSK: 0
I20260926 22:47:38.500301 31023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Generated new TSK 1
I20260926 22:47:38.500381 31023 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 22:47:38.561784 31012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:38.562004 31012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:38.562042 31012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:38.563192 31012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:38.563251 31012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:38.563292 31012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:47:38.564646 31012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:45677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=44451
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:38.564889 31012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:38.565104 31012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:38.570768 31038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.570779 31037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.570977 31040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:38.570979 31012 server_base.cc:1034] running on GCE node
I20260926 22:47:38.571198 31012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:38.571435 31012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:38.572599 31012 hybrid_clock.cc:648] HybridClock initialized: now 1790462858572584 us; error 31 us; skew 500 ppm
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:38.575307 31012 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:47:38.575803 31012 webserver.cc:460] Webserver started at http://127.14.184.193:44451/ using document root <none> and password file <none>
I20260926 22:47:38.576071 31012 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:38.576146 31012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:38.577818 31012 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:38.578531 31047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.578737 31012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:38.578827 31012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "99e89883330b4024ab6d4a2c970bc2f7"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "7fdb22a4da658fd095e0f3cf99c60928"
server_key_iv: "36673c1d46d7e15986b49b6ed35b748b"
server_key_version: "encryptionkey@0"
I20260926 22:47:38.579214 31012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:38.613263 31012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:38.616267 31012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:38.616456 31012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:38.616784 31012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:38.617252 31054 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 22:47:38.618646 31012 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 22:47:38.618738 31012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 22:47:38.618772 31012 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 22:47:38.619673 31012 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 22:47:38.619724 31012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:38.619824 31054 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap starting.
I20260926 22:47:38.627308 31012 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:45677
I20260926 22:47:38.627390 31161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:45677 every 8 connection(s)
I20260926 22:47:38.627905 31012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:47:38.630112 31054 log.cc:824] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:38.632627 31054 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:38.632965 31054 tablet_bootstrap.cc:492] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap complete.
I20260926 22:47:38.634109 31054 ts_tablet_manager.cc:1397] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260926 22:47:38.634303 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31012
I20260926 22:47:38.634593 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:44035
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:43523
--webserver_port=40911
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:38.636299 31054 raft_consensus.cc:348] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.636543 31054 raft_consensus.cc:729] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e89883330b4024ab6d4a2c970bc2f7, State: Initialized, Role: FOLLOWER
I20260926 22:47:38.636718 31054 consensus_queue.cc:260] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.637285 31054 ts_tablet_manager.cc:1428] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Time spent starting tablet: real 0.003s user 0.004s sys 0.002s
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:38.642850 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.628721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:49667 (local address 127.14.184.254:43523)
0926 22:47:38.631310 (+ 2589us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.631320 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.631358 (+ 38us) server_negotiation.cc:407] Connection header received
0926 22:47:38.631503 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.631510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.631564 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.631709 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:38.632570 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.634004 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.635717 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.636393 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.637039 (+ 646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.639106 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:38.639110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:38.639110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:38.639120 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:38.640870 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.641392 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.641394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.641394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.641499 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.641799 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.641799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.641799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.642063 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:38.642136 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.642509 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.642617 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2532,"thread_start_us":78,"threads_started":1}
I20260926 22:47:38.642926 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.628642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:49667)
0926 22:47:38.630302 (+ 1660us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:38.630350 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:38.630945 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:38.631781 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:38.631785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:38.632150 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:38.632426 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.632433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.634121 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.634122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.635609 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.635612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.636519 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.636521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.636702 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.637273 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:38.637290 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:38.639002 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:38.640962 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.640969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.640972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.641297 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.641571 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.641572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.641572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.641717 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.642135 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:38.642143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:38.642319 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 22:47:38.642445 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 22:47:38.642631 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1605,"thread_start_us":108,"threads_started":1}
I20260926 22:47:38.643654 31162 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43523
I20260926 22:47:38.643769 31162 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:38.643961 31162 heartbeater.cc:511] Master 127.14.184.254:43523 requested a full tablet report, sending...
I20260926 22:47:38.644722 30974 ts_manager.cc:194] Registered new tserver with Master: 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677)
I20260926 22:47:38.646000 30974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:49667
I20260926 22:47:38.647047 31162 heartbeater.cc:503] Master 127.14.184.254:43523 was elected leader, sending a full tablet report...
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:47:38.653760 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.645397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.14.184.193:45677)
0926 22:47:38.645548 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.645571 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.645750 (+ 179us) server_negotiation.cc:407] Connection header received
0926 22:47:38.645877 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.645880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.645894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.645962 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:38.646713 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.647810 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.648587 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.649018 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.649088 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.651151 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:38.651158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:38.651158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:38.651165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:38.652527 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.652955 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.652957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.652958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.653025 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.653223 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.653224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.653224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.653365 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:38.653432 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.653598 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.653700 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260926 22:47:38.653759 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.645343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.0.0.1:50800)
0926 22:47:38.645635 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:38.645675 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:38.645770 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:38.645961 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:38.645965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:38.646522 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:38.646601 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.646604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.647903 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.647904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.648493 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.648495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.649094 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.649094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.649149 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.649587 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:38.649591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:38.651077 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:38.652612 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.652618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.652620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.652859 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.653098 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.653098 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.653099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.653160 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.653428 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:38.653434 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:38.653482 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 22:47:38.653560 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:47:38.653665 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":86,"threads_started":1}
I20260926 22:47:38.654227 31096 tablet_service.cc:1514] Processing DeleteTablet for tablet a250035f950547f6ae963fd94f1400fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:18 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:50800
I20260926 22:47:38.654835 31171 tablet_replica.cc:330] stopping tablet replica
I20260926 22:47:38.654954 31171 raft_consensus.cc:2229] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:38.655036 31171 raft_consensus.cc:2258] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:38.655323 31171 ts_tablet_manager.cc:1905] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:47:38.657382 31171 ts_tablet_manager.cc:1918] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:47:38.657445 31171 log.cc:1196] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal/wals/a250035f950547f6ae963fd94f1400fc
I20260926 22:47:38.657701 31171 ts_tablet_manager.cc:1939] T a250035f950547f6ae963fd94f1400fc P 99e89883330b4024ab6d4a2c970bc2f7: Deleting consensus metadata
I20260926 22:47:38.658191 30960 catalog_manager.cc:4841] TS 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677): tablet a250035f950547f6ae963fd94f1400fc (table test-table [id=1bb771c7087248acbe08f6a161dc105a]) successfully deleted
W20260926 22:47:38.718724 31167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:38.718953 31167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:38.718990 31167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:38.720141 31167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:38.720209 31167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:38.720254 31167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:47:38.721707 31167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:44035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=40911
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:38.721962 31167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:38.722184 31167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:38.727102 31179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.727110 31177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.727113 31176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:38.727420 31167 server_base.cc:1034] running on GCE node
I20260926 22:47:38.727671 31167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:38.727946 31167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:38.729102 31167 hybrid_clock.cc:648] HybridClock initialized: now 1790462858729082 us; error 35 us; skew 500 ppm
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:38.731456 31167 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:47:38.731959 31167 webserver.cc:460] Webserver started at http://127.14.184.194:40911/ using document root <none> and password file <none>
I20260926 22:47:38.732179 31167 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:38.732259 31167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:38.733539 31167 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:47:38.734102 31186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.734258 31167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:47:38.734338 31167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "9c77c86a5c323d29ec4017fc6f31487d"
server_key_iv: "343112204d2c18293c39401a4707e42d"
server_key_version: "encryptionkey@0"
I20260926 22:47:38.734647 31167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:38.764129 31167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:38.766479 31167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:38.766640 31167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:38.766940 31167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:38.767261 31167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:38.767295 31167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.767318 31167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:38.767364 31167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.772907 31167 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:44035
I20260926 22:47:38.773011 31299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:44035 every 8 connection(s)
I20260926 22:47:38.773367 31167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:38.780398 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31167
I20260926 22:47:38.780663 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:33955
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:43523
--webserver_port=45017
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:38.782774 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.774113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:39659 (local address 127.14.184.254:43523)
0926 22:47:38.774179 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.774181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.774377 (+ 196us) server_negotiation.cc:407] Connection header received
0926 22:47:38.775016 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.775017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.775031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.775087 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:38.775999 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.776500 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.777426 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.777631 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.777688 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.779879 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:38.779880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:38.779880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:38.779887 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:38.781335 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.781804 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.781804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.781805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.781850 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.782119 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.782120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.782121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.782232 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:38.782303 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.782566 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.782650 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:47:38.782936 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.774099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:39659)
0926 22:47:38.774254 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:38.774287 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:38.774898 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:38.775156 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:38.775160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:38.775704 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:38.775893 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.775897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.776576 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.776576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.777347 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.777349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.777703 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.777704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.777805 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.778313 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:38.778324 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:38.779793 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:38.781411 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.781417 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.781419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.781730 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.781933 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.781934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.781934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.782044 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.782295 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:38.782301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:38.782434 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 22:47:38.782534 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 22:47:38.782711 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260926 22:47:38.783435 31300 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43523
I20260926 22:47:38.783533 31300 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:38.783706 31300 heartbeater.cc:511] Master 127.14.184.254:43523 requested a full tablet report, sending...
I20260926 22:47:38.784094 30974 ts_manager.cc:194] Registered new tserver with Master: 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035)
I20260926 22:47:38.784623 30974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:39659
W20260926 22:47:38.864084 31303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:38.864274 31303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:38.864296 31303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:38.865379 31303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:38.865414 31303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:38.865439 31303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:47:38.866804 31303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:33955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=45017
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:38.866967 31303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:38.867125 31303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:47:38.871934 31303 server_base.cc:1034] running on GCE node
W20260926 22:47:38.871909 31311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.871899 31309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:38.871883 31308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:38.872308 31303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:38.872540 31303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:38.873687 31303 hybrid_clock.cc:648] HybridClock initialized: now 1790462858873674 us; error 28 us; skew 500 ppm
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:38.876168 31303 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:47:38.876585 31303 webserver.cc:460] Webserver started at http://127.14.184.195:45017/ using document root <none> and password file <none>
I20260926 22:47:38.876780 31303 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:38.876829 31303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:38.878077 31303 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:47:38.878643 31318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.878857 31303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:47:38.878926 31303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "bb545787c354421198422a4336dad159"
format_stamp: "Formatted at 2026-09-26 22:47:18 on dist-test-slave-3986"
server_key: "adb1eb727fe57b4856e958ab320ecf91"
server_key_iv: "ba0a7023c686624a3398ebeaa77f8bf0"
server_key_version: "encryptionkey@0"
I20260926 22:47:38.879211 31303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:38.897121 31303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:38.899379 31303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:38.899480 31303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:38.899674 31303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:38.899905 31303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:38.899933 31303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.899955 31303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:38.899966 31303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.905617 31303 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33955
I20260926 22:47:38.905694 31431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33955 every 8 connection(s)
I20260926 22:47:38.906069 31303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
Sep 26 22:47:38 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462858, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:47:38.915091 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.906806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:38809 (local address 127.14.184.254:43523)
0926 22:47:38.906918 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.906921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.907136 (+ 215us) server_negotiation.cc:407] Connection header received
0926 22:47:38.907566 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.907567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.907577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.907649 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:38.908489 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.908988 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.909979 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.910175 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.910228 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.912673 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:38.912675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:38.912675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:38.912681 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:38.913929 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.914266 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.914267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.914267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.914317 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.914517 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.914517 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.914517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.914608 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:38.914727 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.914965 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.915027 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 22:47:38.915232 31434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.906804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:38809)
0926 22:47:38.907012 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:38.907045 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:38.907478 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:38.907700 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:38.907703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:38.908224 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:47:38.908400 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.908404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.909069 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.909069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.909903 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.909905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.910243 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.910244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.910333 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.910902 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:47:38.910911 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:47:38.912564 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:47:38.913997 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.914002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.914004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.914198 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.914374 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:47:38.914374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:47:38.914375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:47:38.914463 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:47:38.914694 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:47:38.914698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:47:38.914841 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 22:47:38.914929 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 22:47:38.915037 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":150,"threads_started":1}
I20260926 22:47:38.915657 31432 heartbeater.cc:347] Connected to a master server at 127.14.184.254:43523
I20260926 22:47:38.915733 31432 heartbeater.cc:464] Registering TS with master...
I20260926 22:47:38.915858 31432 heartbeater.cc:511] Master 127.14.184.254:43523 requested a full tablet report, sending...
I20260926 22:47:38.916193 30974 ts_manager.cc:194] Registered new tserver with Master: bb545787c354421198422a4336dad159 (127.14.184.195:33955)
I20260926 22:47:38.916271 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31303
I20260926 22:47:38.916687 30974 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:38809
I20260926 22:47:38.922610 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.916668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.14.184.254:43523)
0926 22:47:38.916719 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.916721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.916947 (+ 226us) server_negotiation.cc:407] Connection header received
0926 22:47:38.917024 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.917025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.917034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.917083 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:38.917640 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.918085 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.918954 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.919150 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.919223 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.920400 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:38.920401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:38.920401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:38.920408 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:38.921520 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.921854 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.921855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.921855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.921909 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:38.922125 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:38.922125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:38.922126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:38.922228 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:38.922408 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.922444 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.922515 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 22:47:38.923360 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:47:38.925830 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.923803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44330 (local address 127.14.184.254:43523)
0926 22:47:38.923858 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.923859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.923868 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:47:38.923899 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.923900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.923930 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.923991 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:47:38.924061 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.924532 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.925219 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.925396 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.925454 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.925564 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:47:38.925640 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:47:38.925715 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.925729 (+ 14us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.925740 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:47:38.926054 30974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:47:38.927677 30974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:47:38.929750 31096 tablet_service.cc:1467] Processing CreateTablet for tablet af1f5f62cfeb48d493ad5a19abf65189 (DEFAULT_TABLE table=test-table [id=f61abbdf80d54e7082291f3ca065c386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:38.929950 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1f5f62cfeb48d493ad5a19abf65189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:38.931551 31054 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap starting.
I20260926 22:47:38.931857 31054 tablet_bootstrap.cc:654] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:38.932816 31054 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: No bootstrap required, opened a new log
I20260926 22:47:38.932874 31054 ts_tablet_manager.cc:1397] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 22:47:38.933032 31054 raft_consensus.cc:348] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.933105 31054 raft_consensus.cc:374] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:38.933178 31054 raft_consensus.cc:729] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e89883330b4024ab6d4a2c970bc2f7, State: Initialized, Role: FOLLOWER
I20260926 22:47:38.933233 31054 consensus_queue.cc:260] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.933288 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.929261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:44035 (local address 127.0.0.1:54076)
0926 22:47:38.929332 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:38.929344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:38.929388 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:38.929814 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:38.929815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:38.929816 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:38.929871 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.929873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.931136 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.931137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.932092 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.932095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.933027 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.933028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.933099 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.933120 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 22:47:38.933181 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 22:47:38.933221 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":44}
I20260926 22:47:38.933331 31054 ts_tablet_manager.cc:1428] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:38.933342 31438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.929400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.14.184.194:44035)
0926 22:47:38.929608 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.929612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.929622 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:47:38.929691 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.929694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.929709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.929802 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:38.929961 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.931035 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.932186 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.932953 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.933032 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.933089 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.933212 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.933258 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260926 22:47:38.933990 31234 tablet_service.cc:1467] Processing CreateTablet for tablet af1f5f62cfeb48d493ad5a19abf65189 (DEFAULT_TABLE table=test-table [id=f61abbdf80d54e7082291f3ca065c386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:38.934304 31234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1f5f62cfeb48d493ad5a19abf65189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:38.937007 31441 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Bootstrap starting.
I20260926 22:47:38.937426 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.929365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33955 (local address 127.0.0.1:48692)
0926 22:47:38.929603 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:38.929612 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:38.929660 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:38.930164 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:38.930165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:38.930167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:38.930208 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.930210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.931375 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.931376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.932584 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:38.932587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.937157 (+ 4570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.937158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:38.937251 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.937254 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:38.937309 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:47:38.937350 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":60,"threads_started":1}
I20260926 22:47:38.937438 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:38.929448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48692 (local address 127.14.184.195:33955)
0926 22:47:38.929945 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:38.929949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:38.929959 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:47:38.929993 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:38.929995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:38.930010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:38.930084 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:38.930312 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.931291 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.932693 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:38.933357 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:38.937063 (+ 3706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:38.937123 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:38.937324 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 22:47:38.937365 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":81,"threads_started":1}
I20260926 22:47:38.937764 31441 tablet_bootstrap.cc:654] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:38.938365 31441 log.cc:824] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:38.938335 31366 tablet_service.cc:1467] Processing CreateTablet for tablet af1f5f62cfeb48d493ad5a19abf65189 (DEFAULT_TABLE table=test-table [id=f61abbdf80d54e7082291f3ca065c386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:47:38.938633 31366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1f5f62cfeb48d493ad5a19abf65189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:47:38.939079 31441 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: No bootstrap required, opened a new log
I20260926 22:47:38.939131 31441 ts_tablet_manager.cc:1397] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:38.940469 31441 raft_consensus.cc:348] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.940598 31441 raft_consensus.cc:374] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:38.940619 31441 raft_consensus.cc:729] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f047af5bc242d4a715aaf2bb0d80a0, State: Initialized, Role: FOLLOWER
I20260926 22:47:38.940748 31441 consensus_queue.cc:260] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.941025 31441 ts_tablet_manager.cc:1428] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:38.941095 31300 heartbeater.cc:503] Master 127.14.184.254:43523 was elected leader, sending a full tablet report...
I20260926 22:47:38.941627 31444 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Bootstrap starting.
I20260926 22:47:38.942231 31444 tablet_bootstrap.cc:654] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Neither blocks nor log segments found. Creating new log.
I20260926 22:47:38.942767 31444 log.cc:824] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:38.943470 31444 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: No bootstrap required, opened a new log
I20260926 22:47:38.943549 31444 ts_tablet_manager.cc:1397] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:38.944787 31444 raft_consensus.cc:348] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.944931 31444 raft_consensus.cc:374] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:47:38.944993 31444 raft_consensus.cc:729] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb545787c354421198422a4336dad159, State: Initialized, Role: FOLLOWER
I20260926 22:47:38.945118 31444 consensus_queue.cc:260] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:38.945408 31444 ts_tablet_manager.cc:1428] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:47:38.945493 31432 heartbeater.cc:503] Master 127.14.184.254:43523 was elected leader, sending a full tablet report...
W20260926 22:47:39.024092 31301 tablet.cc:2369] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:47:39.072014 31168 raft_consensus.cc:482] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:47:39.072148 31168 raft_consensus.cc:504] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:39.072517 31168 leader_election.cc:288] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035), bb545787c354421198422a4336dad159 (127.14.184.195:33955)
I20260926 22:47:39.075810 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.072748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:44035 (local address 127.14.184.193:49521)
0926 22:47:39.072811 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:39.072824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:39.072927 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:39.073110 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:39.073111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:39.073113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:39.073185 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:39.073188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.074068 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.074069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:39.074974 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:39.074976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.075521 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.075522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:39.075594 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.075625 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 22:47:39.075670 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 22:47:39.075725 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 22:47:39.075819 31438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.072870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:49521 (local address 127.14.184.194:44035)
0926 22:47:39.072923 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:39.072926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:39.072938 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:47:39.073006 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:39.073007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:39.073018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:39.073094 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:39.073277 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.073998 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.075054 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.075443 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.075500 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.075507 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:39.075701 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 22:47:39.075744 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 22:47:39.075843 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.072977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59623 (local address 127.14.184.195:33955)
0926 22:47:39.073030 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:39.073032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:39.073090 (+ 58us) server_negotiation.cc:407] Connection header received
0926 22:47:39.073180 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:39.073181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:39.073190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:39.073246 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:47:39.073430 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.074087 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.075098 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.075464 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.075556 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.075564 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:39.075739 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 22:47:39.075778 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 22:47:39.075824 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.072872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33955 (local address 127.14.184.193:59623)
0926 22:47:39.073041 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:39.073051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:39.073104 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:39.073265 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:47:39.073266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:47:39.073268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:47:39.073347 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:39.073349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.074173 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.074174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:39.074992 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:47:39.074995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.075585 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.075586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:47:39.075636 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.075639 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:47:39.075717 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 22:47:39.075752 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
I20260926 22:47:39.076225 31386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1f5f62cfeb48d493ad5a19abf65189" candidate_uuid: "99e89883330b4024ab6d4a2c970bc2f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb545787c354421198422a4336dad159" is_pre_election: true
I20260926 22:47:39.076289 31254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1f5f62cfeb48d493ad5a19abf65189" candidate_uuid: "99e89883330b4024ab6d4a2c970bc2f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" is_pre_election: true
I20260926 22:47:39.076391 31386 raft_consensus.cc:2454] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e89883330b4024ab6d4a2c970bc2f7 in term 0.
I20260926 22:47:39.076406 31254 raft_consensus.cc:2454] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e89883330b4024ab6d4a2c970bc2f7 in term 0.
I20260926 22:47:39.076619 31050 leader_election.cc:302] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99e89883330b4024ab6d4a2c970bc2f7, bb545787c354421198422a4336dad159; no voters:
I20260926 22:47:39.076746 31168 raft_consensus.cc:2781] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:47:39.076838 31168 raft_consensus.cc:482] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:47:39.076871 31168 raft_consensus.cc:3037] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:39.077566 31168 raft_consensus.cc:504] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:39.077709 31168 leader_election.cc:288] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 1 election: Requested vote from peers 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035), bb545787c354421198422a4336dad159 (127.14.184.195:33955)
I20260926 22:47:39.077943 31254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1f5f62cfeb48d493ad5a19abf65189" candidate_uuid: "99e89883330b4024ab6d4a2c970bc2f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
I20260926 22:47:39.077975 31386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1f5f62cfeb48d493ad5a19abf65189" candidate_uuid: "99e89883330b4024ab6d4a2c970bc2f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb545787c354421198422a4336dad159"
I20260926 22:47:39.078043 31386 raft_consensus.cc:3037] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:39.078045 31254 raft_consensus.cc:3037] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:47:39.078984 31254 raft_consensus.cc:2454] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e89883330b4024ab6d4a2c970bc2f7 in term 1.
I20260926 22:47:39.078984 31386 raft_consensus.cc:2454] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e89883330b4024ab6d4a2c970bc2f7 in term 1.
I20260926 22:47:39.079165 31050 leader_election.cc:302] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f047af5bc242d4a715aaf2bb0d80a0, 99e89883330b4024ab6d4a2c970bc2f7; no voters:
I20260926 22:47:39.079253 31168 raft_consensus.cc:2781] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:47:39.079371 31168 raft_consensus.cc:686] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 LEADER]: Becoming Leader. State: Replica: 99e89883330b4024ab6d4a2c970bc2f7, State: Running, Role: LEADER
I20260926 22:47:39.079460 31168 consensus_queue.cc:237] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:39.080201 30974 catalog_manager.cc:5495] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "99e89883330b4024ab6d4a2c970bc2f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260926 22:47:39.114408 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.112272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.14.184.193:45677)
0926 22:47:39.112337 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:39.112338 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:39.112347 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:47:39.112388 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:39.112389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:39.112407 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:39.112469 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:47:39.112589 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.113074 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.113801 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.114005 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.114058 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.114197 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:47:39.114244 (+ 47us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 22:47:39.114356 (+ 112us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50816: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 22:47:39.114491 31170 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50816: token expired
I20260926 22:47:39.120265 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.114893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.14.184.254:43523)
0926 22:47:39.114956 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:39.114958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:39.114966 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:47:39.114989 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:39.114990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:39.115003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:39.115047 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:47:39.115491 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.115934 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.116561 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.116734 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.116781 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.117901 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:47:39.117902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:47:39.117902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:47:39.117908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:47:39.119365 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:39.119682 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:39.119683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:39.119683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:39.119731 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:47:39.119900 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:47:39.119901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:47:39.119901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:47:39.119991 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:47:39.120063 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:39.120143 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 22:47:39.120203 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 22:47:39.123350 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.121131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.14.184.193:45677)
0926 22:47:39.121168 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:47:39.121170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:47:39.121210 (+ 40us) server_negotiation.cc:407] Connection header received
0926 22:47:39.121239 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:47:39.121239 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:47:39.121247 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:47:39.121304 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:47:39.121395 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.121839 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.122543 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:47:39.122796 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:47:39.122874 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:47:39.123018 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:47:39.123132 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:47:39.123207 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:47:39.123259 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 22:47:39.123276 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:47:39.124976 31386 raft_consensus.cc:1261] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Refusing update from remote peer 99e89883330b4024ab6d4a2c970bc2f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:47:39.124977 31254 raft_consensus.cc:1261] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Refusing update from remote peer 99e89883330b4024ab6d4a2c970bc2f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:47:39.125344 31448 consensus_queue.cc:1035] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:47:39.125474 31168 consensus_queue.cc:1035] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:47:39.127763 31451 mvcc.cc:205] Tried to move back new op lower bound from 7333735870973018112 to 7333735870790029312. Current Snapshot: MvccSnapshot[applied={T|T < 7333735870973018112}]
I20260926 22:47:39.127967 31452 mvcc.cc:205] Tried to move back new op lower bound from 7333735870973018112 to 7333735870790029312. Current Snapshot: MvccSnapshot[applied={T|T < 7333735870973018112}]
I20260926 22:47:39.128530 31453 mvcc.cc:205] Tried to move back new op lower bound from 7333735870973018112 to 7333735870790029312. Current Snapshot: MvccSnapshot[applied={T|T < 7333735870973018112}]
I20260926 22:47:39.129870 30974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:47:39.129956 30974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:47:39.130681 30974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Sending DeleteTablet for 3 replicas of tablet af1f5f62cfeb48d493ad5a19abf65189
I20260926 22:47:39.130998 31096 tablet_service.cc:1514] Processing DeleteTablet for tablet af1f5f62cfeb48d493ad5a19abf65189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:39 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:50800
I20260926 22:47:39.130985 31234 tablet_service.cc:1514] Processing DeleteTablet for tablet af1f5f62cfeb48d493ad5a19abf65189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:39 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:54076
I20260926 22:47:39.131157 31171 tablet_replica.cc:330] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: stopping tablet replica
I20260926 22:47:39.131176 31366 tablet_service.cc:1514] Processing DeleteTablet for tablet af1f5f62cfeb48d493ad5a19abf65189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:47:39 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:48692
I20260926 22:47:39.131229 31171 raft_consensus.cc:2229] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:47:39.131325 31171 raft_consensus.cc:2258] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:39.131349 31461 tablet_replica.cc:330] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: stopping tablet replica
I20260926 22:47:39.131425 31461 raft_consensus.cc:2229] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:39.131454 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 31012
I20260926 22:47:39.131518 31461 raft_consensus.cc:2258] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:39.132292 31461 ts_tablet_manager.cc:1905] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:47:39.134210 31461 ts_tablet_manager.cc:1918] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:47:39.134287 31461 log.cc:1196] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/af1f5f62cfeb48d493ad5a19abf65189
I20260926 22:47:39.134574 31461 ts_tablet_manager.cc:1939] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Deleting consensus metadata
I20260926 22:47:39.135037 30960 catalog_manager.cc:4841] TS bb545787c354421198422a4336dad159 (127.14.184.195:33955): tablet af1f5f62cfeb48d493ad5a19abf65189 (table test-table [id=f61abbdf80d54e7082291f3ca065c386]) successfully deleted
W20260926 22:47:39.138088 30960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:45677 (error 108)
I20260926 22:47:39.138243 31460 tablet_replica.cc:330] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: stopping tablet replica
I20260926 22:47:39.138334 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 31167
I20260926 22:47:39.138365 31460 raft_consensus.cc:2229] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:47:39.138463 31460 raft_consensus.cc:2258] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:47:39.142776 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 31303
I20260926 22:47:39.142797 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.142574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.0.0.1:50830)
0926 22:47:39.142670 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:39.142762 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:45677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 22:47:39.142920 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:39.142817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:44035 (local address 127.0.0.1:54080)
0926 22:47:39.142862 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:39.142884 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.194:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:47:39.144122 30960 catalog_manager.cc:4584] TS 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af1f5f62cfeb48d493ad5a19abf65189: Network error: Client connection negotiation failed: client connection to 127.14.184.193:45677: connect: Connection refused (error 111)
I20260926 22:47:39.147297 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 30941
I20260926 22:47:59.152575 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43523
--webserver_interface=127.14.184.254
--webserver_port=34373
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:47:59.230824 31463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:59.231055 31463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:59.231091 31463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:59.232208 31463 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 22:47:59.232282 31463 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 22:47:59.232304 31463 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 22:47:59.232323 31463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:47:59.232354 31463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:59.232380 31463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:59.232412 31463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:47:59.232443 31463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:47:59.233886 31463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=34373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:59.234146 31463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:59.234373 31463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:59.239034 31469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.239039 31471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.239029 31468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:59.239264 31463 server_base.cc:1034] running on GCE node
I20260926 22:47:59.239524 31463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:59.239840 31463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:59.240981 31463 hybrid_clock.cc:648] HybridClock initialized: now 1790462879240967 us; error 31 us; skew 500 ppm
Sep 26 22:47:59 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:59.248090 31463 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:47:59.248518 31463 webserver.cc:460] Webserver started at http://127.14.184.254:34373/ using document root <none> and password file <none>
I20260926 22:47:59.248759 31463 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:59.248831 31463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:59.250178 31463 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:59.250903 31478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:59.251071 31463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:59.251161 31463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "70be308833824315bea28f9439dc6391"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "ecb344d0b60de69d756c6d9a00a06e9a"
server_key_iv: "8616cb97e114e073579b5ba510cdc395"
server_key_version: "encryptionkey@0"
I20260926 22:47:59.251562 31463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:59.273413 31463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:59.275314 31463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:59.275470 31463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:59.279412 31463 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43523
I20260926 22:47:59.279448 31530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43523 every 8 connection(s)
I20260926 22:47:59.279805 31463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:47:59.281224 31531 sys_catalog.cc:263] Verifying existing consensus state
I20260926 22:47:59.281808 31531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap starting.
I20260926 22:47:59.287581 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31463
I20260926 22:47:59.287858 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:45677
--local_ip_for_outbound_sockets=127.14.184.193
--tserver_master_addrs=127.14.184.254:43523
--webserver_port=44451
--webserver_interface=127.14.184.193
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:59.290179 31531 log.cc:824] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:59.292639 31531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:59.292986 31531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Bootstrap complete.
I20260926 22:47:59.295321 31531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:59.295542 31531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70be308833824315bea28f9439dc6391, State: Initialized, Role: FOLLOWER
I20260926 22:47:59.295718 31531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:59.295800 31531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:47:59.295848 31531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:47:59.295897 31531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 2 FOLLOWER]: Advancing to term 3
I20260926 22:47:59.296873 31531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:59.296958 31531 leader_election.cc:302] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70be308833824315bea28f9439dc6391; no voters:
I20260926 22:47:59.297163 31531 leader_election.cc:288] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 22:47:59.297221 31535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 22:47:59.297426 31531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:47:59.297447 31535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [term 3 LEADER]: Becoming Leader. State: Replica: 70be308833824315bea28f9439dc6391, State: Running, Role: LEADER
I20260926 22:47:59.297627 31535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } }
I20260926 22:47:59.297940 31536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "70be308833824315bea28f9439dc6391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } } }
I20260926 22:47:59.298012 31536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:59.298151 31537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70be308833824315bea28f9439dc6391. Latest consensus state: current_term: 3 leader_uuid: "70be308833824315bea28f9439dc6391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70be308833824315bea28f9439dc6391" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43523 } } }
I20260926 22:47:59.298228 31537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391 [sys.catalog]: This master's current role is: LEADER
I20260926 22:47:59.298333 31543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:47:59.299096 31543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:47:59.299434 31543 catalog_manager.cc:1259] Loaded cluster ID: 4682a95dfc2a456fbf78f3c0a16afbbd
I20260926 22:47:59.299471 31543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:47:59.300676 31543 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:47:59.300838 31543 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Loaded TSK: 0
I20260926 22:47:59.300882 31543 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70be308833824315bea28f9439dc6391: Loaded TSK: 1
F20260926 22:47:59.301086 31543 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f95735839ad google::LogMessage::Fail() at ??:0
@ 0x7f957358688a google::LogMessage::SendToLog() at ??:0
@ 0x7f95735834de google::LogMessage::Flush() at ??:0
@ 0x7f9573584a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9575cf0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9575cedd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f957b4dd1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f957b4dd48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f957b4dd594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f957b4a3c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f957b4adad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f95754510d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9575446954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f95762aa6db start_thread at ??:0
@ 0x7f957208161f clone at ??:0
W20260926 22:47:59.385339 31533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:59.385566 31533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:59.385602 31533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:59.386828 31533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:59.386886 31533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:59.386929 31533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:47:59.388407 31533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:45677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=44451
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:59.388677 31533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:59.388918 31533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:59.393887 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.393919 31573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.393959 31575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:59.394280 31533 server_base.cc:1034] running on GCE node
I20260926 22:47:59.394518 31533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:59.394822 31533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:59.395982 31533 hybrid_clock.cc:648] HybridClock initialized: now 1790462879395967 us; error 31 us; skew 500 ppm
Sep 26 22:47:59 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:59.398447 31533 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:47:59.398950 31533 webserver.cc:460] Webserver started at http://127.14.184.193:44451/ using document root <none> and password file <none>
I20260926 22:47:59.399238 31533 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:59.399307 31533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:59.400974 31533 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 22:47:59.402711 31582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:59.403036 31533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 22:47:59.403112 31533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "99e89883330b4024ab6d4a2c970bc2f7"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "7fdb22a4da658fd095e0f3cf99c60928"
server_key_iv: "36673c1d46d7e15986b49b6ed35b748b"
server_key_version: "encryptionkey@0"
I20260926 22:47:59.403419 31533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:59.416463 31533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:59.418859 31533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:59.419013 31533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:59.419261 31533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:59.419721 31589 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 22:47:59.421737 31533 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 22:47:59.421811 31533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 22:47:59.421856 31533 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 22:47:59.422482 31533 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 22:47:59.422538 31533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:59.422616 31589 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap starting.
I20260926 22:47:59.429728 31533 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:45677
I20260926 22:47:59.429884 31696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:45677 every 8 connection(s)
I20260926 22:47:59.430154 31533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:47:59.433074 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31533
I20260926 22:47:59.433359 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:44035
--local_ip_for_outbound_sockets=127.14.184.194
--tserver_master_addrs=127.14.184.254:43523
--webserver_port=40911
--webserver_interface=127.14.184.194
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:59.433554 31589 log.cc:824] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:59.436420 31589 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:59.436719 31589 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Bootstrap complete.
I20260926 22:47:59.437985 31589 ts_tablet_manager.cc:1397] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.012s
I20260926 22:47:59.439435 31589 raft_consensus.cc:348] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:59.439747 31589 raft_consensus.cc:729] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e89883330b4024ab6d4a2c970bc2f7, State: Initialized, Role: FOLLOWER
I20260926 22:47:59.439918 31589 consensus_queue.cc:260] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:59.440177 31589 ts_tablet_manager.cc:1428] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260926 22:47:59.498049 31699 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:47:59.430833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:59887)
0926 22:47:59.431001 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.431044 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:47:59.431481 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:47:59.497812 (+ 66331us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
W20260926 22:47:59.498356 31697 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43523 (0 consecutive failures): Network error: Failed to ping master at 127.14.184.254:43523: Client connection negotiation failed: client connection to 127.14.184.254:43523: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 22:47:59.498775 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.498605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:46899)
0926 22:47:59.498664 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.498735 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:47:59.499132 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.499032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:39553)
0926 22:47:59.499080 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.499110 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 22:47:59.499261 31697 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 22:47:59.515924 31701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:59.516098 31701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:59.516119 31701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:59.517220 31701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:59.517261 31701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:59.517293 31701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:47:59.518760 31701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:44035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=40911
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:59.519006 31701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:59.519234 31701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:59.524024 31710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:59.524161 31701 server_base.cc:1034] running on GCE node
W20260926 22:47:59.524039 31707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.524055 31708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:59.524463 31701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:59.524695 31701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:59.525848 31701 hybrid_clock.cc:648] HybridClock initialized: now 1790462879525832 us; error 45 us; skew 500 ppm
Sep 26 22:47:59 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:59.528255 31701 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:47:59.528602 31701 webserver.cc:460] Webserver started at http://127.14.184.194:40911/ using document root <none> and password file <none>
I20260926 22:47:59.528787 31701 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:59.528828 31701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:59.530074 31701 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:59.530748 31717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:59.530907 31701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:59.530984 31701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
format_stamp: "Formatted at 2026-09-26 22:47:17 on dist-test-slave-3986"
server_key: "9c77c86a5c323d29ec4017fc6f31487d"
server_key_iv: "343112204d2c18293c39401a4707e42d"
server_key_version: "encryptionkey@0"
I20260926 22:47:59.531260 31701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:59.554957 31701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:59.556654 31701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:59.556757 31701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:59.556921 31701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:59.557344 31724 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 22:47:59.558739 31701 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 22:47:59.558779 31701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 22:47:59.558804 31701 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 22:47:59.559491 31701 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 22:47:59.559523 31701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 22:47:59.559619 31724 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Bootstrap starting.
I20260926 22:47:59.565999 31701 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:44035
I20260926 22:47:59.566071 31831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:44035 every 8 connection(s)
I20260926 22:47:59.566407 31701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:47:59.567515 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.567042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:52051)
0926 22:47:59.567239 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.567345 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
W20260926 22:47:59.567728 31832 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43523 (0 consecutive failures): Network error: Failed to ping master at 127.14.184.254:43523: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
I20260926 22:47:59.568027 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.567932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:37067)
0926 22:47:59.567980 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.568007 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:47:59.568327 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.568263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:60225)
0926 22:47:59.568296 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.568315 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260926 22:47:59.568444 31832 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 22:47:59.569422 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31701
I20260926 22:47:59.569677 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:33955
--local_ip_for_outbound_sockets=127.14.184.195
--tserver_master_addrs=127.14.184.254:43523
--webserver_port=45017
--webserver_interface=127.14.184.195
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:47:59.570243 31724 log.cc:824] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:47:59.572281 31724 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:47:59.572566 31724 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Bootstrap complete.
I20260926 22:47:59.573477 31724 ts_tablet_manager.cc:1397] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260926 22:47:59.574608 31724 raft_consensus.cc:348] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:59.574957 31724 raft_consensus.cc:729] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f047af5bc242d4a715aaf2bb0d80a0, State: Initialized, Role: FOLLOWER
I20260926 22:47:59.575090 31724 consensus_queue.cc:260] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:47:59.575323 31724 ts_tablet_manager.cc:1428] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260926 22:47:59.575590 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.575489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:40801)
0926 22:47:59.575539 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.575564 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:47:59.575868 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.575769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:42457)
0926 22:47:59.575827 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.575849 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 22:47:59.648763 31836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:47:59.648984 31836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:47:59.649024 31836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:47:59.650159 31836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:47:59.650219 31836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:47:59.650264 31836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:47:59.652264 31836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:33955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=45017
--tserver_master_addrs=127.14.184.254:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:47:59.652547 31836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:47:59.652829 31836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:47:59.658089 31842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.658098 31845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:47:59.658115 31843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:47:59.658457 31836 server_base.cc:1034] running on GCE node
I20260926 22:47:59.658702 31836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:47:59.658957 31836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:47:59.660096 31836 hybrid_clock.cc:648] HybridClock initialized: now 1790462879660076 us; error 42 us; skew 500 ppm
Sep 26 22:47:59 dist-test-slave-3986 krb5kdc[30396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:47:59.662497 31836 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:47:59.662948 31836 webserver.cc:460] Webserver started at http://127.14.184.195:45017/ using document root <none> and password file <none>
I20260926 22:47:59.663179 31836 fs_manager.cc:359] Metadata directory not provided
I20260926 22:47:59.663244 31836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:47:59.664556 31836 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:47:59.665153 31852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:59.665342 31836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:47:59.665433 31836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "bb545787c354421198422a4336dad159"
format_stamp: "Formatted at 2026-09-26 22:47:18 on dist-test-slave-3986"
server_key: "adb1eb727fe57b4856e958ab320ecf91"
server_key_iv: "ba0a7023c686624a3398ebeaa77f8bf0"
server_key_version: "encryptionkey@0"
I20260926 22:47:59.665746 31836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:47:59.702836 31836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:47:59.704682 31836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:47:59.704854 31836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:47:59.705093 31836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:47:59.705413 31836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:47:59.705468 31836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:59.705509 31836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:47:59.705538 31836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:47:59.711799 31836 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:33955
I20260926 22:47:59.711844 31965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:33955 every 8 connection(s)
I20260926 22:47:59.712185 31836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:47:59.713189 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.712815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:57503)
0926 22:47:59.712964 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.713040 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
W20260926 22:47:59.713583 31966 heartbeater.cc:644] Failed to heartbeat to 127.14.184.254:43523 (0 consecutive failures): Network error: Failed to ping master at 127.14.184.254:43523: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
I20260926 22:47:59.713892 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.713790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:52765)
0926 22:47:59.713841 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.713869 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:47:59.714187 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:47:59.714084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:49173)
0926 22:47:59.714129 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:47:59.714162 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 22:47:59.714285 31966 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 22:47:59.714650 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 31836
I20260926 22:48:00.499866 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.499611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:59629)
0926 22:48:00.499739 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:00.499775 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260926 22:48:00.576409 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.576215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:52269)
0926 22:48:00.576333 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:00.576362 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260926 22:48:00.714996 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.714773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:35925)
0926 22:48:00.714912 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:00.714947 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":55,"threads_started":1}
I20260926 22:48:00.882155 31973 raft_consensus.cc:482] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:00.882269 31973 raft_consensus.cc:504] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:48:00.882718 31973 leader_election.cc:288] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035), bb545787c354421198422a4336dad159 (127.14.184.195:33955)
Sep 26 22:48:00 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.195@KRBTEST.COM
Sep 26 22:48:00 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.194@KRBTEST.COM
I20260926 22:48:00.893095 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.883052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:51453 (local address 127.14.184.194:44035)
0926 22:48:00.883193 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:00.883206 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:00.883232 (+ 26us) server_negotiation.cc:407] Connection header received
0926 22:48:00.883363 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:00.883364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:00.883406 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:00.883506 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:00.884403 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.885369 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.886360 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.886804 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.886911 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:00.889790 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:00.889794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:00.889794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:00.889803 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:00.891675 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:00.892019 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:00.892021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:00.892021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:00.892130 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:00.892323 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:00.892324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:00.892325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:00.892610 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:00.892666 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:00.892870 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 22:48:00.893001 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260926 22:48:00.893106 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.883156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:55701 (local address 127.14.184.195:33955)
0926 22:48:00.883320 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:00.883331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:00.883366 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:48:00.883975 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:00.883978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:00.884026 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:00.884161 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:00.884486 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.885291 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.886382 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.886815 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.886940 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:00.889341 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:00.889345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:00.889345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:00.889355 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:00.891399 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:00.891824 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:00.891827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:00.891827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:00.891906 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:00.892150 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:00.892151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:00.892151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:00.892469 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:00.892550 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:00.892880 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 22:48:00.892999 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260926 22:48:00.893096 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.883050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:33955 (local address 127.14.184.193:55701)
0926 22:48:00.883177 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:00.883187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:00.883265 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:00.884150 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:00.884151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:00.884302 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:00.884387 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:00.884389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.885392 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.885393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:00.886267 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:00.886271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.886911 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.886912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:00.887023 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:00.887630 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:00.887635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:00.889240 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:00.891486 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:00.891492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:00.891494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:00.891729 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:00.891972 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:00.891973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:00.891974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:00.892084 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:00.892538 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:00.892544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:00.892685 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 22:48:00.892806 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 22:48:00.892983 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":71,"thread_start_us":61,"threads_started":1}
I20260926 22:48:00.893143 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.882904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:44035 (local address 127.14.184.193:51453)
0926 22:48:00.882972 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:00.882986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:00.883057 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:00.883563 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:00.883566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:00.884006 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:00.884283 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:00.884293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.885468 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.885469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:00.886266 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:00.886268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:00.886887 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:00.886888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:00.887023 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:00.887609 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:00.887619 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:00.889693 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:00.891747 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:00.891748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:00.891749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:00.891925 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:00.892198 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:00.892199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:00.892199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:00.892247 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:00.892670 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:00.892671 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:00.892701 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 22:48:00.892799 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 22:48:00.893036 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":135}
W20260926 22:48:00.893816 31585 leader_election.cc:341] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb545787c354421198422a4336dad159 (127.14.184.195:33955): Not found: Tablet not found: af1f5f62cfeb48d493ad5a19abf65189
I20260926 22:48:00.893976 31786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1f5f62cfeb48d493ad5a19abf65189" candidate_uuid: "99e89883330b4024ab6d4a2c970bc2f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" is_pre_election: true
I20260926 22:48:00.894129 31786 raft_consensus.cc:2454] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e89883330b4024ab6d4a2c970bc2f7 in term 1.
I20260926 22:48:00.894371 31585 leader_election.cc:302] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15f047af5bc242d4a715aaf2bb0d80a0, 99e89883330b4024ab6d4a2c970bc2f7; no voters: bb545787c354421198422a4336dad159
I20260926 22:48:00.894482 31973 raft_consensus.cc:2781] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 22:48:00.894541 31973 raft_consensus.cc:482] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:00.894563 31973 raft_consensus.cc:3037] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 1 FOLLOWER]: Advancing to term 2
I20260926 22:48:00.895661 31973 raft_consensus.cc:504] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:48:00.895802 31973 leader_election.cc:288] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 2 election: Requested vote from peers 15f047af5bc242d4a715aaf2bb0d80a0 (127.14.184.194:44035), bb545787c354421198422a4336dad159 (127.14.184.195:33955)
I20260926 22:48:00.895962 31786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1f5f62cfeb48d493ad5a19abf65189" candidate_uuid: "99e89883330b4024ab6d4a2c970bc2f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "15f047af5bc242d4a715aaf2bb0d80a0"
I20260926 22:48:00.896046 31786 raft_consensus.cc:3037] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 1 FOLLOWER]: Advancing to term 2
W20260926 22:48:00.896117 31585 leader_election.cc:341] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer bb545787c354421198422a4336dad159 (127.14.184.195:33955): Not found: Tablet not found: af1f5f62cfeb48d493ad5a19abf65189
I20260926 22:48:00.897221 31786 raft_consensus.cc:2454] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e89883330b4024ab6d4a2c970bc2f7 in term 2.
I20260926 22:48:00.897379 31585 leader_election.cc:302] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15f047af5bc242d4a715aaf2bb0d80a0, 99e89883330b4024ab6d4a2c970bc2f7; no voters: bb545787c354421198422a4336dad159
I20260926 22:48:00.897496 31973 raft_consensus.cc:2781] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 22:48:00.897645 31973 raft_consensus.cc:686] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [term 2 LEADER]: Becoming Leader. State: Replica: 99e89883330b4024ab6d4a2c970bc2f7, State: Running, Role: LEADER
I20260926 22:48:00.897753 31973 consensus_queue.cc:237] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:48:00.897941 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:00.897816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:44453)
0926 22:48:00.897880 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:00.897913 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 22:48:01.326249 31786 raft_consensus.cc:1261] T af1f5f62cfeb48d493ad5a19abf65189 P 15f047af5bc242d4a715aaf2bb0d80a0 [term 2 FOLLOWER]: Refusing update from remote peer 99e89883330b4024ab6d4a2c970bc2f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260926 22:48:01.326707 31973 consensus_queue.cc:1035] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260926 22:48:01.327766 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.327510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:45675)
0926 22:48:01.327620 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.327719 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":77,"threads_started":1}
I20260926 22:48:01.328395 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.328293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:44357)
0926 22:48:01.328337 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.328363 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 22:48:01.328500 31585 consensus_peers.cc:487] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7 -> Peer bb545787c354421198422a4336dad159 (127.14.184.195:33955): Couldn't send request to peer bb545787c354421198422a4336dad159. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af1f5f62cfeb48d493ad5a19abf65189. This is attempt 1: this message will repeat every 5th retry.
I20260926 22:48:01.328603 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.328507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:40729)
0926 22:48:01.328547 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.328572 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 22:48:01.715664 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.715463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:44857)
0926 22:48:01.715589 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.715625 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
I20260926 22:48:01.820873 31985 ts_tablet_manager.cc:927] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Initiating tablet copy from peer 99e89883330b4024ab6d4a2c970bc2f7 (127.14.184.193:45677)
I20260926 22:48:01.821099 31985 tablet_copy_client.cc:316] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: tablet copy: Beginning tablet copy session from remote peer at address 127.14.184.193:45677
Sep 26 22:48:01 dist-test-slave-3986 krb5kdc[30396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462879, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.193@KRBTEST.COM
I20260926 22:48:01.829028 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.821382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:45677 (local address 127.14.184.195:58717)
0926 22:48:01.821460 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.821481 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:01.821563 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:01.821786 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:01.821791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:01.822080 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:01.822153 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:01.822155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:01.823110 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:01.823111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:01.823704 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:01.823705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:01.824150 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:01.824151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:01.824216 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:01.824729 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:01.824733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:01.826052 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:01.827893 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:01.827899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:01.827901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:01.828087 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:01.828319 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:01.828320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:01.828320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:01.828396 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:01.828731 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:01.828736 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:01.828784 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 22:48:01.828864 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 22:48:01.828949 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 22:48:01.829128 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.821481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:58717 (local address 127.14.184.193:45677)
0926 22:48:01.821621 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:01.821626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:01.821636 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:01.821661 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:01.821663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:01.821700 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:01.821773 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:01.822240 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:01.823030 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:01.823781 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:01.824072 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:01.824127 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:01.826140 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:01.826148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:01.826148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:01.826155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:01.827825 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:01.828166 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:01.828169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:01.828169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:01.828242 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:01.828485 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:01.828486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:01.828486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:01.828645 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:01.828758 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:01.828943 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 22:48:01.829016 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260926 22:48:01.829563 31671 tablet_copy_service.cc:140] P 99e89883330b4024ab6d4a2c970bc2f7: Received BeginTabletCopySession request for tablet af1f5f62cfeb48d493ad5a19abf65189 from peer bb545787c354421198422a4336dad159 ({username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:58717)
I20260926 22:48:01.829643 31671 tablet_copy_service.cc:161] P 99e89883330b4024ab6d4a2c970bc2f7: Beginning new tablet copy session on tablet af1f5f62cfeb48d493ad5a19abf65189 from peer bb545787c354421198422a4336dad159 at {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:58717: session id = bb545787c354421198422a4336dad159-af1f5f62cfeb48d493ad5a19abf65189
I20260926 22:48:01.830142 31671 tablet_copy_source_session.cc:205] T af1f5f62cfeb48d493ad5a19abf65189 P 99e89883330b4024ab6d4a2c970bc2f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 22:48:01.831159 31985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1f5f62cfeb48d493ad5a19abf65189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:01.833712 31985 tablet_copy_client.cc:791] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: tablet copy: Starting download of 0 data blocks...
I20260926 22:48:01.833909 31985 tablet_copy_client.cc:655] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: tablet copy: Starting download of 1 WAL segments...
I20260926 22:48:01.834630 31985 tablet_copy_client.cc:531] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 22:48:01.835341 31985 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Bootstrap starting.
I20260926 22:48:01.836658 31985 log.cc:824] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:01.839090 31985 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 22:48:01.839319 31985 tablet_bootstrap.cc:492] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Bootstrap complete.
I20260926 22:48:01.839458 31985 ts_tablet_manager.cc:1397] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260926 22:48:01.840385 31985 raft_consensus.cc:348] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:48:01.840528 31985 raft_consensus.cc:729] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb545787c354421198422a4336dad159, State: Initialized, Role: FOLLOWER
I20260926 22:48:01.840669 31985 consensus_queue.cc:260] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f047af5bc242d4a715aaf2bb0d80a0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 44035 } } peers { permanent_uuid: "bb545787c354421198422a4336dad159" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 33955 } } peers { permanent_uuid: "99e89883330b4024ab6d4a2c970bc2f7" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 45677 } }
I20260926 22:48:01.840929 31985 ts_tablet_manager.cc:1428] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260926 22:48:01.841192 31671 tablet_copy_service.cc:342] P 99e89883330b4024ab6d4a2c970bc2f7: Request end of tablet copy session bb545787c354421198422a4336dad159-af1f5f62cfeb48d493ad5a19abf65189 received from {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:58717
I20260926 22:48:01.841249 31671 tablet_copy_service.cc:433] P 99e89883330b4024ab6d4a2c970bc2f7: ending tablet copy session bb545787c354421198422a4336dad159-af1f5f62cfeb48d493ad5a19abf65189 on tablet af1f5f62cfeb48d493ad5a19abf65189 with peer bb545787c354421198422a4336dad159
I20260926 22:48:01.841253 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.841127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:34565)
0926 22:48:01.841183 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.841223 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 22:48:01.841521 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:01.841444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:51021)
0926 22:48:01.841475 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:01.841500 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 22:48:02.329138 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:02.328886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:43849)
0926 22:48:02.329053 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:02.329093 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260926 22:48:02.329180 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:02.328996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:42363)
0926 22:48:02.329105 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:02.329136 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":73,"threads_started":1}
I20260926 22:48:02.395469 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:02.395332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:37303)
0926 22:48:02.395392 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:02.395426 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 22:48:02.395493 31919 raft_consensus.cc:1203] T af1f5f62cfeb48d493ad5a19abf65189 P bb545787c354421198422a4336dad159 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260926 22:48:02.396029 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:02.395864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:33025)
0926 22:48:02.395961 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:02.395990 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":63,"threads_started":1}
I20260926 22:48:03.329869 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:03.329666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:50661)
0926 22:48:03.329780 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:03.329822 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260926 22:48:03.396054 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:03.395854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:42137)
0926 22:48:03.395978 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:03.396009 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260926 22:48:03.396595 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:03.396409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:41755)
0926 22:48:03.396523 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:03.396560 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260926 22:48:04.330493 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:04.330290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:39047)
0926 22:48:04.330409 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:04.330447 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260926 22:48:04.396643 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:04.396448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:43851)
0926 22:48:04.396563 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:04.396596 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260926 22:48:04.397269 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:04.397074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:37489)
0926 22:48:04.397196 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:04.397230 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260926 22:48:05.331178 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:05.330962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:36421)
0926 22:48:05.331087 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:05.331123 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260926 22:48:05.397482 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:05.397265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:47555)
0926 22:48:05.397399 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:05.397436 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260926 22:48:05.397893 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:05.397795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:34555)
0926 22:48:05.397845 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:05.397873 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:48:05.398015 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:05.397791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:60415)
0926 22:48:05.397910 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:05.397967 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260926 22:48:06.331774 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:06.331566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:36665)
0926 22:48:06.331684 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:06.331718 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260926 22:48:06.398635 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:06.398411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:41983)
0926 22:48:06.398550 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:06.398589 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":57,"threads_started":1}
I20260926 22:48:06.398653 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:06.398439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:42971)
0926 22:48:06.398579 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:06.398612 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260926 22:48:07.332551 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:07.332338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:41601)
0926 22:48:07.332468 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:07.332505 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260926 22:48:07.399327 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:07.399104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:39659)
0926 22:48:07.399232 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:07.399269 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260926 22:48:07.399345 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:07.399132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:47457)
0926 22:48:07.399272 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:07.399306 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260926 22:48:08.333206 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:08.333003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:44049)
0926 22:48:08.333123 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:08.333160 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260926 22:48:08.400048 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:08.399835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:48189)
0926 22:48:08.399960 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:08.399985 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260926 22:48:08.400074 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:08.399836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:36505)
0926 22:48:08.399968 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:08.400007 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260926 22:48:09.333894 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:09.333681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:48019)
0926 22:48:09.333814 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:09.333847 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260926 22:48:09.400848 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:09.400602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:54367)
0926 22:48:09.400757 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:09.400805 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260926 22:48:09.400847 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:09.400636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:36031)
0926 22:48:09.400771 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:09.400801 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":54,"threads_started":1}
I20260926 22:48:10.334615 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:10.334377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:57727)
0926 22:48:10.334530 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:10.334569 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260926 22:48:10.401607 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:10.401377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:36041)
0926 22:48:10.401514 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:10.401561 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260926 22:48:10.401607 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:10.401410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:55235)
0926 22:48:10.401529 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:10.401564 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260926 22:48:11.335248 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:11.335054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:34323)
0926 22:48:11.335169 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:11.335203 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260926 22:48:11.402321 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:11.402099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:46581)
0926 22:48:11.402239 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:11.402277 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260926 22:48:11.402379 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:11.402149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:57689)
0926 22:48:11.402298 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:11.402341 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260926 22:48:12.335848 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:12.335646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:59097)
0926 22:48:12.335771 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:12.335804 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260926 22:48:12.403040 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:12.402828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:55619)
0926 22:48:12.402966 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:12.402995 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260926 22:48:12.403039 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:12.402835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:37215)
0926 22:48:12.402972 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:12.402997 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260926 22:48:13.336452 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:13.336263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:37821)
0926 22:48:13.336375 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:13.336407 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260926 22:48:13.403699 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:13.403479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:54965)
0926 22:48:13.403597 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:13.403634 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260926 22:48:13.403700 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:13.403513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:36653)
0926 22:48:13.403631 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:13.403660 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260926 22:48:14.337117 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:14.336911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.194:40417)
0926 22:48:14.337035 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:14.337070 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260926 22:48:14.404397 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:14.404194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.195:48207)
0926 22:48:14.404312 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:14.404345 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260926 22:48:14.404420 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:14.404191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:43523 (local address 127.14.184.193:36363)
0926 22:48:14.404352 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:14.404380 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.254:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260926 22:48:14.716737 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 31533
I20260926 22:48:14.721361 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 31701
I20260926 22:48:14.725666 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 31836
2026-09-26T22:48:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58346 ms)
[----------] 30 tests from SecurityITest (226249 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:14 dist-test-slave-3986 krb5kdc[32035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:14 dist-test-slave-3986 krb5kdc[32035](info): set up 2 sockets
Sep 26 22:48:14 dist-test-slave-3986 krb5kdc[32035](info): commencing operation
krb5kdc: starting...
W20260926 22:48:15.754570 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:15 dist-test-slave-3986 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:15.867697 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:35141
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:35141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:15.946892 32048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:15.947088 32048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:15.947109 32048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:15.948266 32048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:15.948305 32048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:15.948319 32048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:15.948333 32048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:15.948369 32048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:15.949841 32048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:35141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:35141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:15.950088 32048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:15.950320 32048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:15.955490 32056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:15.955495 32053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:15.955500 32054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:15.955639 32048 server_base.cc:1034] running on GCE node
I20260926 22:48:15.956075 32048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:15.956370 32048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:15.957510 32048 hybrid_clock.cc:648] HybridClock initialized: now 1790462895957500 us; error 27 us; skew 500 ppm
Sep 26 22:48:15 dist-test-slave-3986 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462895, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:15.959997 32048 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:15.960434 32048 webserver.cc:460] Webserver started at http://127.14.184.254:41467/ using document root <none> and password file <none>
I20260926 22:48:15.960667 32048 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:15.960741 32048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:15.960886 32048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:15.961899 32048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "2acaca505ca24e699b81bfad9a635073"
format_stamp: "Formatted at 2026-09-26 22:48:15 on dist-test-slave-3986"
server_key: "a83792d2f12c23fa06420d38485e3c35"
server_key_iv: "68f7233b4901b146b19447ed3712d2ba"
server_key_version: "encryptionkey@0"
I20260926 22:48:15.962249 32048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "2acaca505ca24e699b81bfad9a635073"
format_stamp: "Formatted at 2026-09-26 22:48:15 on dist-test-slave-3986"
server_key: "a83792d2f12c23fa06420d38485e3c35"
server_key_iv: "68f7233b4901b146b19447ed3712d2ba"
server_key_version: "encryptionkey@0"
I20260926 22:48:15.963593 32048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:15.964298 32063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:15.964466 32048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:15.964555 32048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "2acaca505ca24e699b81bfad9a635073"
format_stamp: "Formatted at 2026-09-26 22:48:15 on dist-test-slave-3986"
server_key: "a83792d2f12c23fa06420d38485e3c35"
server_key_iv: "68f7233b4901b146b19447ed3712d2ba"
server_key_version: "encryptionkey@0"
I20260926 22:48:15.964634 32048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:15.973932 32048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:15.975663 32048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:15.975821 32048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:15.979635 32048 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:35141
I20260926 22:48:15.979667 32115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:35141 every 8 connection(s)
I20260926 22:48:15.980002 32048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:15.980790 32116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:15.981736 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 32048
I20260926 22:48:15.981848 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:15.982050 15075 external_mini_cluster.cc:1428] Setting key 821db8f8db0609d02c6827126274161f
I20260926 22:48:15.983479 32116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: Bootstrap starting.
I20260926 22:48:15.984050 32116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:15.984349 32116 log.cc:824] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:15.985033 32116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: No bootstrap required, opened a new log
I20260926 22:48:15.986891 32116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acaca505ca24e699b81bfad9a635073" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 35141 } }
I20260926 22:48:15.987056 32116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:15.987142 32116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acaca505ca24e699b81bfad9a635073, State: Initialized, Role: FOLLOWER
I20260926 22:48:15.987324 32116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acaca505ca24e699b81bfad9a635073" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 35141 } }
I20260926 22:48:15.987426 32116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:15.987471 32116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:15.987519 32116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:15.988142 32116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acaca505ca24e699b81bfad9a635073" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 35141 } }
I20260926 22:48:15.988263 32116 leader_election.cc:302] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acaca505ca24e699b81bfad9a635073; no voters:
I20260926 22:48:15.988415 32116 leader_election.cc:288] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:15.988476 32121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:15.988664 32116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:15.988663 32121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [term 1 LEADER]: Becoming Leader. State: Replica: 2acaca505ca24e699b81bfad9a635073, State: Running, Role: LEADER
Sep 26 22:48:15 dist-test-slave-3986 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:15.988866 32121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acaca505ca24e699b81bfad9a635073" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 35141 } }
I20260926 22:48:15.989334 32123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acaca505ca24e699b81bfad9a635073. Latest consensus state: current_term: 1 leader_uuid: "2acaca505ca24e699b81bfad9a635073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acaca505ca24e699b81bfad9a635073" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 35141 } } }
I20260926 22:48:15.989786 32123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:15.990058 32122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2acaca505ca24e699b81bfad9a635073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acaca505ca24e699b81bfad9a635073" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 35141 } } }
I20260926 22:48:15.990310 32122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073 [sys.catalog]: This master's current role is: LEADER
W20260926 22:48:15.990787 32137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:15.990847 32137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:15.990942 32135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:15.991117 32135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:15.992785 32135 catalog_manager.cc:1347] Generated new cluster ID: 3133a542ccc7420ead3b38e6ef249ca3
I20260926 22:48:15.992836 32135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:15.993089 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:15.982558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.14.184.254:35141)
0926 22:48:15.982761 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:15.982771 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:15.982802 (+ 31us) server_negotiation.cc:407] Connection header received
0926 22:48:15.983313 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:15.983319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:15.983363 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:15.983513 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:15.984146 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:15.985305 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:15.986067 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:15.986474 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:15.986591 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:15.989349 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:15.989353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:15.989354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:15.989366 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:15.991411 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:15.991840 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:15.991843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:15.991844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:15.991956 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:15.992176 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:15.992177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:15.992178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:15.992489 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:15.992660 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:15.992765 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 22:48:15.992926 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260926 22:48:16.011756 32135 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:16.012269 32135 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:16.020298 32135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: Generated new TSK 0
I20260926 22:48:16.020483 32135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:48:16.050114 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:35141
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:16.145469 32142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:16.145664 32142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:16.145686 32142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:16.146842 32142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:16.146881 32142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:16.146907 32142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:48:16.148272 32142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:35141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:16.148450 32142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:16.148612 32142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:16.153501 32150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.153525 32148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.153615 32147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:16.153625 32142 server_base.cc:1034] running on GCE node
I20260926 22:48:16.153970 32142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:16.154214 32142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:16.155375 32142 hybrid_clock.cc:648] HybridClock initialized: now 1790462896155365 us; error 43 us; skew 500 ppm
Sep 26 22:48:16 dist-test-slave-3986 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462896, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:16.157713 32142 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:48:16.158200 32142 webserver.cc:460] Webserver started at http://127.14.184.193:43901/ using document root <none> and password file <none>
I20260926 22:48:16.158421 32142 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:16.158490 32142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:16.158643 32142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:16.159674 32142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "8b37a36ce66c450d9b3b8f38aafd3d19"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "d13c7c45157855fcbb413e53790c6fff"
server_key_iv: "d81a915f78b12706c9fd37fc586c87e2"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.160039 32142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "8b37a36ce66c450d9b3b8f38aafd3d19"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "d13c7c45157855fcbb413e53790c6fff"
server_key_iv: "d81a915f78b12706c9fd37fc586c87e2"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.161346 32142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:48:16.162106 32157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.162324 32142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:16.162421 32142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "8b37a36ce66c450d9b3b8f38aafd3d19"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "d13c7c45157855fcbb413e53790c6fff"
server_key_iv: "d81a915f78b12706c9fd37fc586c87e2"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.162505 32142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:16.174240 32142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:16.176036 32142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:16.176192 32142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:16.176405 32142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:16.176690 32142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:16.176751 32142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.176790 32142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:16.176816 32142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.182482 32142 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:40877
I20260926 22:48:16.182530 32270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:40877 every 8 connection(s)
I20260926 22:48:16.182885 32142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:48:16.185231 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 32142
I20260926 22:48:16.185324 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:48:16.185534 15075 external_mini_cluster.cc:1428] Setting key fb16566f3f527fd6916b1479532645d5
Sep 26 22:48:16 dist-test-slave-3986 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462896, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:16.193104 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.183566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:33441 (local address 127.14.184.254:35141)
0926 22:48:16.183628 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.183630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.183851 (+ 221us) server_negotiation.cc:407] Connection header received
0926 22:48:16.184381 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.184382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.184395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.184456 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:16.185447 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.186141 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.187195 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.187540 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.187607 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.190116 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:16.190117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:16.190118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:16.190126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:16.191780 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.192154 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.192155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.192156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.192218 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.192440 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.192441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.192441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.192586 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:16.192673 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.192904 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.192995 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:48:16.193110 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.183553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:35141 (local address 127.14.184.193:33441)
0926 22:48:16.183719 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.183762 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.184275 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.184524 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.184528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.185122 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:16.185359 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.185366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.186239 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.186240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.187106 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.187108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.187640 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.187641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.187753 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.188295 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:16.188303 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:16.190029 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:16.191859 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:16.191866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:16.191867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:16.192076 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:16.192286 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:16.192286 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:16.192286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:16.192377 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:16.192666 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:16.192670 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:16.192778 (+ 108us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.192854 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.192961 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260926 22:48:16.193589 32271 heartbeater.cc:347] Connected to a master server at 127.14.184.254:35141
I20260926 22:48:16.193691 32271 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:16.193856 32271 heartbeater.cc:511] Master 127.14.184.254:35141 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
I20260926 22:48:16.194442 32080 ts_manager.cc:194] Registered new tserver with Master: 8b37a36ce66c450d9b3b8f38aafd3d19 (127.14.184.193:40877)
I20260926 22:48:16.195127 32080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:33441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:48:16.209640 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:35141
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:16.289179 32276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:16.289367 32276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:16.289390 32276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:16.290678 32276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:16.290724 32276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:16.290750 32276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:48:16.292213 32276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:35141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:16.292454 32276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:16.292667 32276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:16.297655 32284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.297696 32281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.297658 32282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:16.297865 32276 server_base.cc:1034] running on GCE node
I20260926 22:48:16.298178 32276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:16.298417 32276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:16.299572 32276 hybrid_clock.cc:648] HybridClock initialized: now 1790462896299566 us; error 36 us; skew 500 ppm
Sep 26 22:48:16 dist-test-slave-3986 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462896, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:16.302104 32276 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:48:16.302553 32276 webserver.cc:460] Webserver started at http://127.14.184.194:33709/ using document root <none> and password file <none>
I20260926 22:48:16.302851 32276 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:16.302919 32276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:16.303068 32276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:16.304133 32276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "14353f7efa824c728bb107bc71135ee5"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "6197bfc02ad1db2c0a7a0784feb00ac8"
server_key_iv: "a55a3303fc491cd320f8ead026b28bb4"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.304469 32276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "14353f7efa824c728bb107bc71135ee5"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "6197bfc02ad1db2c0a7a0784feb00ac8"
server_key_iv: "a55a3303fc491cd320f8ead026b28bb4"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.305749 32276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:16.306463 32291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.306627 32276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:16.306769 32276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "14353f7efa824c728bb107bc71135ee5"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "6197bfc02ad1db2c0a7a0784feb00ac8"
server_key_iv: "a55a3303fc491cd320f8ead026b28bb4"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.306847 32276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:16.317266 32276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:16.319502 32276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:16.319672 32276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:16.319892 32276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:16.320186 32276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:16.320243 32276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.320283 32276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:16.320322 32276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.326047 32276 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:38877
I20260926 22:48:16.326098 32404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:38877 every 8 connection(s)
I20260926 22:48:16.326380 32276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
Sep 26 22:48:16 dist-test-slave-3986 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462896, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:16.334017 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 32276
I20260926 22:48:16.334133 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:48:16.334365 15075 external_mini_cluster.cc:1428] Setting key 4bbd95ea00fbf10620502daed49a20e2
I20260926 22:48:16.334932 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.327075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:47951 (local address 127.14.184.254:35141)
0926 22:48:16.327120 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.327122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.327329 (+ 207us) server_negotiation.cc:407] Connection header received
0926 22:48:16.327766 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.327767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.327776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.327838 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:16.328706 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.329213 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.330144 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.330336 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.330384 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.332312 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:16.332314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:16.332314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:16.332320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:16.333431 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.333789 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.333790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.333790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.333835 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.334149 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.334150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.334150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.334296 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:16.334357 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.334761 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.334843 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:48:16.335112 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.327026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:35141 (local address 127.14.184.194:47951)
0926 22:48:16.327173 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.327212 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.327676 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.327890 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.327894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.328433 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:16.328617 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.328622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.329293 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.329294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.330068 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.330070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.330404 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.330405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.330501 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.331004 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:16.331012 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:16.332233 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:16.333495 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:16.333500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:16.333502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:16.333729 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:16.333900 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:16.333901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:16.333901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:16.334036 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:16.334370 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:16.334376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:16.334556 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.334730 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.334881 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260926 22:48:16.335554 32405 heartbeater.cc:347] Connected to a master server at 127.14.184.254:35141
I20260926 22:48:16.335654 32405 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:16.335851 32405 heartbeater.cc:511] Master 127.14.184.254:35141 requested a full tablet report, sending...
I20260926 22:48:16.336210 32080 ts_manager.cc:194] Registered new tserver with Master: 14353f7efa824c728bb107bc71135ee5 (127.14.184.194:38877)
I20260926 22:48:16.336692 32080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:47951
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:48:16.358894 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:35141
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:16.439244 32410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:16.439431 32410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:16.439453 32410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:16.440488 32410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:16.440523 32410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:16.440548 32410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:48:16.441879 32410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:35141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:16.442046 32410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:16.442200 32410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:16.447247 32418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.447299 32416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:16.447412 32410 server_base.cc:1034] running on GCE node
W20260926 22:48:16.447336 32415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:16.447646 32410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:16.447907 32410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:16.449071 32410 hybrid_clock.cc:648] HybridClock initialized: now 1790462896449052 us; error 36 us; skew 500 ppm
Sep 26 22:48:16 dist-test-slave-3986 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462896, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:16.451448 32410 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:48:16.451915 32410 webserver.cc:460] Webserver started at http://127.14.184.195:45261/ using document root <none> and password file <none>
I20260926 22:48:16.452139 32410 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:16.452205 32410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:16.452365 32410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:16.453356 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "36ed60bdbc0a4d90b2e158fac6c26768"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "670fbbd2813cb1ac367719cebdd3992e"
server_key_iv: "7232c3522cbbc3388851c9543bdf47aa"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.453712 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "36ed60bdbc0a4d90b2e158fac6c26768"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "670fbbd2813cb1ac367719cebdd3992e"
server_key_iv: "7232c3522cbbc3388851c9543bdf47aa"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.455101 32410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:16.455801 32425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.455965 32410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:16.456059 32410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "36ed60bdbc0a4d90b2e158fac6c26768"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "670fbbd2813cb1ac367719cebdd3992e"
server_key_iv: "7232c3522cbbc3388851c9543bdf47aa"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.456137 32410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:16.479382 32410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:16.481240 32410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:16.481413 32410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:16.481643 32410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:16.481940 32410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:16.481999 32410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.482039 32410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:16.482067 32410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.488236 32410 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:44343
I20260926 22:48:16.488273 32538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:44343 every 8 connection(s)
I20260926 22:48:16.488672 32410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:48:16.493491 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 32410
I20260926 22:48:16.493598 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:48:16.493767 15075 external_mini_cluster.cc:1428] Setting key 4d2591f8ab169b861c5d33e497f9b304
Sep 26 22:48:16 dist-test-slave-3986 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462896, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:16.498283 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.489460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:35791 (local address 127.14.184.254:35141)
0926 22:48:16.489500 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.489502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.489694 (+ 192us) server_negotiation.cc:407] Connection header received
0926 22:48:16.490299 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.490300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.490314 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.490385 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:16.491375 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.491893 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.492879 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.493195 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.493266 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.495778 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:16.495780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:16.495780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:16.495787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:16.496926 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.497333 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.497334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.497334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.497393 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.497639 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.497640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.497640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.497756 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:16.497851 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.498113 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.498189 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:48:16.498485 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.489400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:35141 (local address 127.14.184.195:35791)
0926 22:48:16.489570 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.489611 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.490179 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.490451 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.490455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.491088 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:16.491279 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.491284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.491989 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.491990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.492774 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.492776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.493329 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.493330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.493501 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.494009 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:16.494036 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:16.495685 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:16.497012 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:16.497018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:16.497021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:16.497257 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:16.497460 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:16.497460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:16.497461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:16.497566 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:16.497841 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:16.497847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:16.497983 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.498092 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.498252 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260926 22:48:16.498962 32539 heartbeater.cc:347] Connected to a master server at 127.14.184.254:35141
I20260926 22:48:16.499038 32539 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:16.499182 32539 heartbeater.cc:511] Master 127.14.184.254:35141 requested a full tablet report, sending...
I20260926 22:48:16.499475 32080 ts_manager.cc:194] Registered new tserver with Master: 36ed60bdbc0a4d90b2e158fac6c26768 (127.14.184.195:44343)
I20260926 22:48:16.499827 32080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:35791
I20260926 22:48:16.505487 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:48:16.512284 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.506883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.14.184.254:35141)
0926 22:48:16.506918 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.506919 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.507008 (+ 89us) server_negotiation.cc:407] Connection header received
0926 22:48:16.507058 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.507059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.507066 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.507109 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:16.507630 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.508099 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.508789 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.508963 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.509016 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.510138 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:16.510139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:16.510140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:16.510146 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:16.511410 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.511705 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.511706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.511706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.511751 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:16.511908 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:16.511908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:16.511909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:16.512025 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:16.512097 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.512177 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.512230 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:48:16.513707 32080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:48:16.515385 32080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:48:16.521718 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.517710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38877 (local address 127.0.0.1:53912)
0926 22:48:16.518196 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.518206 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.518257 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.518496 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.518496 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.518498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:16.518533 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.518534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.519457 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.519458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.520697 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.520699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.521404 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.521405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.521480 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.521493 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.521585 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.521638 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":58,"threads_started":1}
I20260926 22:48:16.521734 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.517806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.14.184.194:38877)
0926 22:48:16.517966 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.517971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.518259 (+ 288us) server_negotiation.cc:407] Connection header received
0926 22:48:16.518323 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.518325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.518341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.518412 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:16.518625 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.519354 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.520792 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.521318 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.521406 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.521463 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.521602 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.521656 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260926 22:48:16.521914 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.517503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:44343 (local address 127.0.0.1:34312)
0926 22:48:16.517597 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.517634 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.517689 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.518178 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.518182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.518185 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:16.518251 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.518253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.519814 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.519815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.520759 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.520761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.521649 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.521650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.521712 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.521716 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.521747 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.521810 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":5,"thread_start_us":54,"threads_started":1}
I20260926 22:48:16.522047 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.517531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:40877 (local address 127.0.0.1:44898)
0926 22:48:16.517794 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.517804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.517843 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.518356 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.518357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.518359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:16.518412 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.518414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.519795 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.519796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.520848 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.520850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.521662 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.521663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.521716 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.521718 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.521747 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.521810 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":24,"thread_start_us":61,"threads_started":1}
I20260926 22:48:16.522331 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.517832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.14.184.195:44343)
0926 22:48:16.517972 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.517977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.517987 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:16.518021 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.518023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.518039 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.518155 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:16.518480 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.519689 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.520845 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.521540 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.522042 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.522195 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.522223 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.522260 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260926 22:48:16.522434 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.518015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.14.184.193:40877)
0926 22:48:16.518213 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.518217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.518226 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:16.518264 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.518266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.518281 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.518363 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:16.518524 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.519684 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.520924 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.521565 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.522211 (+ 646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.522271 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.522293 (+ 22us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.522348 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260926 22:48:16.522447 32339 tablet_service.cc:1467] Processing CreateTablet for tablet 5c74917356c840989585a8a858ac2778 (DEFAULT_TABLE table=test-table [id=2fbc8f06e1bc43769a4fd80d2e798dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:16.522845 32339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c74917356c840989585a8a858ac2778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:16.522987 32473 tablet_service.cc:1467] Processing CreateTablet for tablet 5c74917356c840989585a8a858ac2778 (DEFAULT_TABLE table=test-table [id=2fbc8f06e1bc43769a4fd80d2e798dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:16.523223 32473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c74917356c840989585a8a858ac2778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:16.525308 32553 tablet_bootstrap.cc:492] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Bootstrap starting.
I20260926 22:48:16.525539 32554 tablet_bootstrap.cc:492] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Bootstrap starting.
I20260926 22:48:16.525861 32205 tablet_service.cc:1467] Processing CreateTablet for tablet 5c74917356c840989585a8a858ac2778 (DEFAULT_TABLE table=test-table [id=2fbc8f06e1bc43769a4fd80d2e798dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:16.526202 32553 tablet_bootstrap.cc:654] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:16.526219 32205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c74917356c840989585a8a858ac2778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:16.526222 32554 tablet_bootstrap.cc:654] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:16.526759 32553 log.cc:824] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:16.526768 32554 log.cc:824] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:16.527467 32553 tablet_bootstrap.cc:492] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: No bootstrap required, opened a new log
I20260926 22:48:16.527511 32554 tablet_bootstrap.cc:492] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: No bootstrap required, opened a new log
I20260926 22:48:16.527545 32553 ts_tablet_manager.cc:1397] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 22:48:16.527580 32554 ts_tablet_manager.cc:1397] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:48:16.528852 32553 raft_consensus.cc:348] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.529006 32553 raft_consensus.cc:374] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:16.529050 32553 raft_consensus.cc:729] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14353f7efa824c728bb107bc71135ee5, State: Initialized, Role: FOLLOWER
I20260926 22:48:16.529183 32553 consensus_queue.cc:260] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.529470 32553 ts_tablet_manager.cc:1428] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:48:16.529601 32405 heartbeater.cc:503] Master 127.14.184.254:35141 was elected leader, sending a full tablet report...
I20260926 22:48:16.529578 32554 raft_consensus.cc:348] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.529752 32554 raft_consensus.cc:374] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:16.529806 32554 raft_consensus.cc:729] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ed60bdbc0a4d90b2e158fac6c26768, State: Initialized, Role: FOLLOWER
I20260926 22:48:16.529881 32558 tablet_bootstrap.cc:492] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: Bootstrap starting.
I20260926 22:48:16.529927 32554 consensus_queue.cc:260] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.530217 32539 heartbeater.cc:503] Master 127.14.184.254:35141 was elected leader, sending a full tablet report...
I20260926 22:48:16.530224 32554 ts_tablet_manager.cc:1428] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:48:16.530542 32558 tablet_bootstrap.cc:654] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:16.531118 32558 log.cc:824] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:16.531828 32558 tablet_bootstrap.cc:492] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: No bootstrap required, opened a new log
I20260926 22:48:16.531877 32558 ts_tablet_manager.cc:1397] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:16.533195 32558 raft_consensus.cc:348] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.533363 32558 raft_consensus.cc:374] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:16.533386 32558 raft_consensus.cc:729] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b37a36ce66c450d9b3b8f38aafd3d19, State: Initialized, Role: FOLLOWER
I20260926 22:48:16.533489 32558 consensus_queue.cc:260] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.533701 32558 ts_tablet_manager.cc:1428] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:16.533772 32271 heartbeater.cc:503] Master 127.14.184.254:35141 was elected leader, sending a full tablet report...
W20260926 22:48:16.576907 32406 tablet.cc:2369] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:48:16.629020 32561 raft_consensus.cc:482] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:16.629140 32561 raft_consensus.cc:504] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.629567 32561 leader_election.cc:288] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36ed60bdbc0a4d90b2e158fac6c26768 (127.14.184.195:44343), 14353f7efa824c728bb107bc71135ee5 (127.14.184.194:38877)
I20260926 22:48:16.632608 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.629761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:44343 (local address 127.14.184.193:57663)
0926 22:48:16.629835 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.629851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.629910 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.630065 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.630066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.630068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:16.630153 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.630156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.630953 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.630954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.631746 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.631748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.632336 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.632338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.632420 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.632425 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.632476 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.632515 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260926 22:48:16.632613 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.629862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:57663 (local address 127.14.184.195:44343)
0926 22:48:16.629912 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.629914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.629924 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:16.629985 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.629986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.629996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.630057 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:16.630261 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.630881 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.631846 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.632258 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.632314 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.632321 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.632499 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.632542 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 22:48:16.632714 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.629954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40919 (local address 127.14.184.194:38877)
0926 22:48:16.629999 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.630001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.630168 (+ 167us) server_negotiation.cc:407] Connection header received
0926 22:48:16.630199 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.630200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.630212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.630267 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:16.630429 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.630962 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.631946 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.632341 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.632412 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.632419 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.632591 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.632635 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:48:16.632745 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.629907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:38877 (local address 127.14.184.193:40919)
0926 22:48:16.630088 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.630097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.630148 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.630285 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.630285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.630287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:16.630358 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.630359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.631042 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.631043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.631856 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.631858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.632425 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.632425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.632507 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.632511 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.632566 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.632601 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260926 22:48:16.632920 32493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c74917356c840989585a8a858ac2778" candidate_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" is_pre_election: true
I20260926 22:48:16.633049 32493 raft_consensus.cc:2454] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b37a36ce66c450d9b3b8f38aafd3d19 in term 0.
I20260926 22:48:16.633085 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c74917356c840989585a8a858ac2778" candidate_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14353f7efa824c728bb107bc71135ee5" is_pre_election: true
I20260926 22:48:16.633245 32359 raft_consensus.cc:2454] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b37a36ce66c450d9b3b8f38aafd3d19 in term 0.
I20260926 22:48:16.633322 32158 leader_election.cc:302] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ed60bdbc0a4d90b2e158fac6c26768, 8b37a36ce66c450d9b3b8f38aafd3d19; no voters:
I20260926 22:48:16.633476 32561 raft_consensus.cc:2781] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:48:16.633548 32561 raft_consensus.cc:482] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:16.633565 32561 raft_consensus.cc:3037] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:16.634306 32561 raft_consensus.cc:504] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.634431 32561 leader_election.cc:288] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [CANDIDATE]: Term 1 election: Requested vote from peers 36ed60bdbc0a4d90b2e158fac6c26768 (127.14.184.195:44343), 14353f7efa824c728bb107bc71135ee5 (127.14.184.194:38877)
I20260926 22:48:16.634605 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c74917356c840989585a8a858ac2778" candidate_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14353f7efa824c728bb107bc71135ee5"
I20260926 22:48:16.634622 32493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c74917356c840989585a8a858ac2778" candidate_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36ed60bdbc0a4d90b2e158fac6c26768"
I20260926 22:48:16.634738 32359 raft_consensus.cc:3037] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:16.634760 32493 raft_consensus.cc:3037] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:16.635433 32359 raft_consensus.cc:2454] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b37a36ce66c450d9b3b8f38aafd3d19 in term 1.
I20260926 22:48:16.635530 32493 raft_consensus.cc:2454] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b37a36ce66c450d9b3b8f38aafd3d19 in term 1.
I20260926 22:48:16.635596 32158 leader_election.cc:302] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14353f7efa824c728bb107bc71135ee5, 8b37a36ce66c450d9b3b8f38aafd3d19; no voters:
I20260926 22:48:16.635758 32561 raft_consensus.cc:2781] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:16.635931 32561 raft_consensus.cc:686] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 1 LEADER]: Becoming Leader. State: Replica: 8b37a36ce66c450d9b3b8f38aafd3d19, State: Running, Role: LEADER
I20260926 22:48:16.636024 32561 consensus_queue.cc:237] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } }
I20260926 22:48:16.636835 32080 catalog_manager.cc:5495] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b37a36ce66c450d9b3b8f38aafd3d19 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b37a36ce66c450d9b3b8f38aafd3d19" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 40877 } health_report { overall_health: HEALTHY } } }
I20260926 22:48:16.669768 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.667329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44912 (local address 127.14.184.193:40877)
0926 22:48:16.667399 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.667402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.667411 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:16.667478 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.667478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.667496 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.667572 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:16.667760 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.668239 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.668986 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.669182 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.669266 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.669439 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:16.669532 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:16.669618 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.669661 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.669681 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 22:48:16.671551 32493 raft_consensus.cc:1261] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 1 FOLLOWER]: Refusing update from remote peer 8b37a36ce66c450d9b3b8f38aafd3d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:16.671551 32359 raft_consensus.cc:1261] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 1 FOLLOWER]: Refusing update from remote peer 8b37a36ce66c450d9b3b8f38aafd3d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:16.671908 32563 consensus_queue.cc:1035] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14353f7efa824c728bb107bc71135ee5" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 38877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:16.672045 32561 consensus_queue.cc:1035] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36ed60bdbc0a4d90b2e158fac6c26768" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:16.674464 32566 mvcc.cc:205] Tried to move back new op lower bound from 7333736024763891712 to 7333736024621830144. Current Snapshot: MvccSnapshot[applied={T|T < 7333736024763891712}]
I20260926 22:48:16.674738 32567 mvcc.cc:205] Tried to move back new op lower bound from 7333736024763891712 to 7333736024621830144. Current Snapshot: MvccSnapshot[applied={T|T < 7333736024763891712}]
I20260926 22:48:16.674793 32568 mvcc.cc:205] Tried to move back new op lower bound from 7333736024763891712 to 7333736024621830144. Current Snapshot: MvccSnapshot[applied={T|T < 7333736024763891712}]
I20260926 22:48:16.676795 32080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:16.676888 32080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:16.677640 32080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2acaca505ca24e699b81bfad9a635073: Sending DeleteTablet for 3 replicas of tablet 5c74917356c840989585a8a858ac2778
I20260926 22:48:16.678004 32473 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c74917356c840989585a8a858ac2778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:16 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:34312
I20260926 22:48:16.678007 32339 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c74917356c840989585a8a858ac2778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:16 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:53912
I20260926 22:48:16.678203 32205 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c74917356c840989585a8a858ac2778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:16 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:44898
I20260926 22:48:16.678391 32578 tablet_replica.cc:330] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19: stopping tablet replica
I20260926 22:48:16.678491 32578 raft_consensus.cc:2229] T 5c74917356c840989585a8a858ac2778 P 8b37a36ce66c450d9b3b8f38aafd3d19 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:48:16.678493 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 32142
I20260926 22:48:16.679849 32577 tablet_replica.cc:330] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: stopping tablet replica
I20260926 22:48:16.679965 32577 raft_consensus.cc:2229] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:16.680065 32577 raft_consensus.cc:2258] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:16.680238 32576 tablet_replica.cc:330] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: stopping tablet replica
I20260926 22:48:16.680328 32576 raft_consensus.cc:2229] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:16.680428 32576 raft_consensus.cc:2258] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:16.680512 32577 ts_tablet_manager.cc:1905] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:16.680835 32576 ts_tablet_manager.cc:1905] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:16.682243 32577 ts_tablet_manager.cc:1918] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:16.682319 32577 log.cc:1196] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/5c74917356c840989585a8a858ac2778
I20260926 22:48:16.682497 32576 ts_tablet_manager.cc:1918] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:16.682570 32576 log.cc:1196] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/5c74917356c840989585a8a858ac2778
I20260926 22:48:16.682588 32577 ts_tablet_manager.cc:1939] T 5c74917356c840989585a8a858ac2778 P 36ed60bdbc0a4d90b2e158fac6c26768: Deleting consensus metadata
I20260926 22:48:16.682844 32576 ts_tablet_manager.cc:1939] T 5c74917356c840989585a8a858ac2778 P 14353f7efa824c728bb107bc71135ee5: Deleting consensus metadata
I20260926 22:48:16.683076 32064 catalog_manager.cc:4841] TS 36ed60bdbc0a4d90b2e158fac6c26768 (127.14.184.195:44343): tablet 5c74917356c840989585a8a858ac2778 (table test-table [id=2fbc8f06e1bc43769a4fd80d2e798dd2]) successfully deleted
I20260926 22:48:16.683490 32064 catalog_manager.cc:4841] TS 14353f7efa824c728bb107bc71135ee5 (127.14.184.194:38877): tablet 5c74917356c840989585a8a858ac2778 (table test-table [id=2fbc8f06e1bc43769a4fd80d2e798dd2]) successfully deleted
W20260926 22:48:16.684656 32066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:40877 (error 108)
I20260926 22:48:16.684938 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 32276
I20260926 22:48:16.685467 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.684996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:40877 (local address 127.0.0.1:44920)
0926 22:48:16.685393 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.685433 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376}
W20260926 22:48:16.685596 32066 catalog_manager.cc:4584] TS 8b37a36ce66c450d9b3b8f38aafd3d19 (127.14.184.193:40877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c74917356c840989585a8a858ac2778: Network error: Client connection negotiation failed: client connection to 127.14.184.193:40877: connect: Connection refused (error 111)
I20260926 22:48:16.689949 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 32410
I20260926 22:48:16.694455 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 32048
2026-09-26T22:48:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1968 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T22:48:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:16Z Disabled control of system clock
I20260926 22:48:16.721336 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45881
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:45881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:16.800184 32583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:16.800382 32583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:16.800402 32583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:16.801532 32583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:16.801577 32583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:16.801590 32583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:16.801641 32583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:16.801666 32583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:16.803017 32583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:45881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:16.803246 32583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:16.803529 32583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:16.808174 32591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.808184 32589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.808204 32588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:16.808378 32583 server_base.cc:1034] running on GCE node
I20260926 22:48:16.808712 32583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:16.809051 32583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:16.810209 32583 hybrid_clock.cc:648] HybridClock initialized: now 1790462896810187 us; error 44 us; skew 500 ppm
I20260926 22:48:16.811435 32583 webserver.cc:460] Webserver started at http://127.14.184.254:46489/ using document root <none> and password file <none>
I20260926 22:48:16.811702 32583 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:16.811749 32583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:16.811909 32583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:16.812951 32583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "77138a04c4ae4f63a73c06f72e76f31b"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "7cb1c092585dc107d247ee2bff36dff0"
server_key_iv: "c62c5726f02a86865aecae6250fd505d"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.813311 32583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "77138a04c4ae4f63a73c06f72e76f31b"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "7cb1c092585dc107d247ee2bff36dff0"
server_key_iv: "c62c5726f02a86865aecae6250fd505d"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.814738 32583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:16.815621 32597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.815886 32583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:16.815968 32583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "77138a04c4ae4f63a73c06f72e76f31b"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "7cb1c092585dc107d247ee2bff36dff0"
server_key_iv: "c62c5726f02a86865aecae6250fd505d"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.816063 32583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:16.827447 32583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:16.827752 32583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:16.827898 32583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:16.831688 32583 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:45881
I20260926 22:48:16.831722 32649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:45881 every 8 connection(s)
I20260926 22:48:16.832091 32583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:16.833025 32650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:16.835254 32650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b: Bootstrap starting.
I20260926 22:48:16.835457 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 32583
I20260926 22:48:16.835568 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:16.835783 15075 external_mini_cluster.cc:1428] Setting key 569beab87277eb2df86dc401d51cf5da
I20260926 22:48:16.835953 32650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:16.836297 32650 log.cc:824] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:16.836954 32650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b: No bootstrap required, opened a new log
I20260926 22:48:16.838644 32650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77138a04c4ae4f63a73c06f72e76f31b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45881 } }
I20260926 22:48:16.838863 32650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:16.838912 32650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77138a04c4ae4f63a73c06f72e76f31b, State: Initialized, Role: FOLLOWER
I20260926 22:48:16.839033 32650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77138a04c4ae4f63a73c06f72e76f31b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45881 } }
I20260926 22:48:16.839133 32650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:16.839176 32650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:16.839223 32650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:16.839745 32650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77138a04c4ae4f63a73c06f72e76f31b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45881 } }
I20260926 22:48:16.839854 32650 leader_election.cc:302] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77138a04c4ae4f63a73c06f72e76f31b; no voters:
I20260926 22:48:16.840021 32650 leader_election.cc:288] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:16.840061 32655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:16.840202 32655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [term 1 LEADER]: Becoming Leader. State: Replica: 77138a04c4ae4f63a73c06f72e76f31b, State: Running, Role: LEADER
I20260926 22:48:16.840286 32650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:16.840314 32655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77138a04c4ae4f63a73c06f72e76f31b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45881 } }
I20260926 22:48:16.840626 32657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77138a04c4ae4f63a73c06f72e76f31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77138a04c4ae4f63a73c06f72e76f31b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45881 } } }
I20260926 22:48:16.840716 32657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:16.840960 32661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:16.841272 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.836259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.14.184.254:45881)
0926 22:48:16.836494 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.836503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.836532 (+ 29us) server_negotiation.cc:407] Connection header received
0926 22:48:16.836952 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.836958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.837013 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.837177 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:16.837393 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.838259 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.839233 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.839668 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.839760 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.840498 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:16.840502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:16.840503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:16.840702 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:16.840764 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:16.840778 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:16.840839 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.840952 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.841042 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260926 22:48:16.841107 32656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77138a04c4ae4f63a73c06f72e76f31b. Latest consensus state: current_term: 1 leader_uuid: "77138a04c4ae4f63a73c06f72e76f31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77138a04c4ae4f63a73c06f72e76f31b" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45881 } } }
I20260926 22:48:16.841440 32656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:16.842028 32661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:16.843444 32661 catalog_manager.cc:1347] Generated new cluster ID: ad7eed2e020f43a1ad0e3e5499c2aa63
I20260926 22:48:16.843494 32661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:16.853402 32661 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:16.853905 32661 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:16.857618 32661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b: Generated new TSK 0
I20260926 22:48:16.857787 32661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:16.862298 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45881
--builtin_ntp_servers=127.14.184.212:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:16.942745 32674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:16.942957 32674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:16.942981 32674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:16.944123 32674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:16.944162 32674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:16.944186 32674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:48:16.945536 32674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:16.945786 32674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:16.946080 32674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:16.951054 32679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.951053 32682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:16.951090 32680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:16.951332 32674 server_base.cc:1034] running on GCE node
I20260926 22:48:16.951524 32674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:16.951810 32674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:16.952962 32674 hybrid_clock.cc:648] HybridClock initialized: now 1790462896952949 us; error 32 us; skew 500 ppm
I20260926 22:48:16.954174 32674 webserver.cc:460] Webserver started at http://127.14.184.193:43123/ using document root <none> and password file <none>
I20260926 22:48:16.954409 32674 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:16.954450 32674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:16.954535 32674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:16.955687 32674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "bd4893943b4847d9987d9f8f4bba9446"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "b7f4b54b7b2f45082eeabd92404a2782"
server_key_iv: "ed267d61539f5d962d59abff338a4f05"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.956048 32674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "bd4893943b4847d9987d9f8f4bba9446"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "b7f4b54b7b2f45082eeabd92404a2782"
server_key_iv: "ed267d61539f5d962d59abff338a4f05"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.957432 32674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:16.958195 32688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.958381 32674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:16.958467 32674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "bd4893943b4847d9987d9f8f4bba9446"
format_stamp: "Formatted at 2026-09-26 22:48:16 on dist-test-slave-3986"
server_key: "b7f4b54b7b2f45082eeabd92404a2782"
server_key_iv: "ed267d61539f5d962d59abff338a4f05"
server_key_version: "encryptionkey@0"
I20260926 22:48:16.958556 32674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:16.975636 32674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:16.975947 32674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:16.976094 32674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:16.976315 32674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:16.976634 32674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:16.976696 32674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.976742 32674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:16.976768 32674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:16.982301 32674 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:43809
I20260926 22:48:16.982376 333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:43809 every 8 connection(s)
I20260926 22:48:16.982758 32674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:48:16.987185 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 32674
I20260926 22:48:16.987298 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:48:16.987519 15075 external_mini_cluster.cc:1428] Setting key 9dde9f6151056f2204c097b86a600da8
I20260926 22:48:16.987890 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.983491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:56869 (local address 127.14.184.254:45881)
0926 22:48:16.983533 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:16.983535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:16.983786 (+ 251us) server_negotiation.cc:407] Connection header received
0926 22:48:16.984243 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:16.984244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:16.984253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:16.984312 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:16.984934 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.985441 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.986373 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.986556 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.986605 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.987526 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:16.987528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:16.987528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:16.987532 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:16.987546 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:16.987549 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:16.987620 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:16.987763 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 22:48:16.987790 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:16.988131 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:16.983453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45881 (local address 127.14.184.193:56869)
0926 22:48:16.983644 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:16.983693 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:16.984162 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:16.984365 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:16.984368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:16.984376 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:16.984870 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.984873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.985521 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.985522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.986299 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:16.986301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:16.986628 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:16.986628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:16.986756 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:16.987368 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:16.987369 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:16.987411 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:16.987440 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:16.987668 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:16.987675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:16.987678 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:16.987759 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 22:48:16.987874 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260926 22:48:16.988576 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45881
--builtin_ntp_servers=127.14.184.212:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 22:48:16.988759 334 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45881
I20260926 22:48:16.988863 334 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:16.989046 334 heartbeater.cc:511] Master 127.14.184.254:45881 requested a full tablet report, sending...
I20260926 22:48:16.989595 32614 ts_manager.cc:194] Registered new tserver with Master: bd4893943b4847d9987d9f8f4bba9446 (127.14.184.193:43809)
I20260926 22:48:16.990338 32614 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.193:56869
W20260926 22:48:17.068130 337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:17.068351 337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:17.068385 337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:17.069504 337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:17.069566 337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:17.069608 337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:48:17.071003 337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:17.071261 337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:17.071542 337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:17.076354 345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:17.076416 337 server_base.cc:1034] running on GCE node
W20260926 22:48:17.076361 343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:17.076331 342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:17.076799 337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:17.077047 337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:17.078213 337 hybrid_clock.cc:648] HybridClock initialized: now 1790462897078210 us; error 37 us; skew 500 ppm
I20260926 22:48:17.079430 337 webserver.cc:460] Webserver started at http://127.14.184.194:46869/ using document root <none> and password file <none>
I20260926 22:48:17.079661 337 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:17.079743 337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:17.079897 337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:17.080914 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "cefe81b965eb430aa1fc01586d0a5a42"
format_stamp: "Formatted at 2026-09-26 22:48:17 on dist-test-slave-3986"
server_key: "144a2cbf6b15a417bdf6c93247354e6d"
server_key_iv: "b2938ceaf2eb077ad6b5a48b37cbff9f"
server_key_version: "encryptionkey@0"
I20260926 22:48:17.081274 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "cefe81b965eb430aa1fc01586d0a5a42"
format_stamp: "Formatted at 2026-09-26 22:48:17 on dist-test-slave-3986"
server_key: "144a2cbf6b15a417bdf6c93247354e6d"
server_key_iv: "b2938ceaf2eb077ad6b5a48b37cbff9f"
server_key_version: "encryptionkey@0"
I20260926 22:48:17.082543 337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:17.083334 351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:17.083498 337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:17.083601 337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "cefe81b965eb430aa1fc01586d0a5a42"
format_stamp: "Formatted at 2026-09-26 22:48:17 on dist-test-slave-3986"
server_key: "144a2cbf6b15a417bdf6c93247354e6d"
server_key_iv: "b2938ceaf2eb077ad6b5a48b37cbff9f"
server_key_version: "encryptionkey@0"
I20260926 22:48:17.083693 337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:17.095182 337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:17.095461 337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:17.095610 337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:17.095832 337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:17.096136 337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:17.096189 337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:17.096233 337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:17.096259 337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:17.101794 337 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:36615
I20260926 22:48:17.101845 464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:36615 every 8 connection(s)
I20260926 22:48:17.102140 337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:48:17.103242 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 337
I20260926 22:48:17.103363 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:48:17.103595 15075 external_mini_cluster.cc:1428] Setting key 3e600695413f8e3d97dce3186d1f6447
I20260926 22:48:17.104544 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45881
--builtin_ntp_servers=127.14.184.212:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 22:48:17.107460 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.102819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:45019 (local address 127.14.184.254:45881)
0926 22:48:17.102861 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.102862 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.103123 (+ 261us) server_negotiation.cc:407] Connection header received
0926 22:48:17.103692 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.103693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.103713 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.103768 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:17.104436 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.105052 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.106038 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.106251 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.106337 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.107181 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:17.107182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:17.107183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:17.107188 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:17.107204 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:17.107209 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:17.107263 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.107354 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.107383 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:48:17.107666 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.102828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45881 (local address 127.14.184.194:45019)
0926 22:48:17.102992 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.103022 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.103583 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.103838 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.103842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.103849 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:17.104327 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.104331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.105158 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.105159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.105943 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.105945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.106348 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.106349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.106479 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.107040 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:17.107040 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:17.107067 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:17.107089 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:17.107266 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:17.107271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:17.107274 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.107356 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.107457 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260926 22:48:17.108116 465 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45881
I20260926 22:48:17.108219 465 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:17.108390 465 heartbeater.cc:511] Master 127.14.184.254:45881 requested a full tablet report, sending...
I20260926 22:48:17.108783 32614 ts_manager.cc:194] Registered new tserver with Master: cefe81b965eb430aa1fc01586d0a5a42 (127.14.184.194:36615)
I20260926 22:48:17.109344 32614 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.194:45019
W20260926 22:48:17.185051 468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:17.185238 468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:17.185258 468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:17.186390 468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:17.186426 468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:17.186450 468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:48:17.187853 468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:45881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:17.188035 468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:17.188259 468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:17.193110 473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:17.193096 476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:17.193159 474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:17.193209 468 server_base.cc:1034] running on GCE node
I20260926 22:48:17.193478 468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:17.193730 468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:17.194888 468 hybrid_clock.cc:648] HybridClock initialized: now 1790462897194876 us; error 28 us; skew 500 ppm
I20260926 22:48:17.196084 468 webserver.cc:460] Webserver started at http://127.14.184.195:39483/ using document root <none> and password file <none>
I20260926 22:48:17.196319 468 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:17.196384 468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:17.196519 468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:17.197479 468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "fbbf83789fb34b379e298a50c7d264ba"
format_stamp: "Formatted at 2026-09-26 22:48:17 on dist-test-slave-3986"
server_key: "f5dd5c8ec5f813e617fe9daa11c1940b"
server_key_iv: "faea63d0e98a9ce48520b17392819b11"
server_key_version: "encryptionkey@0"
I20260926 22:48:17.197824 468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "fbbf83789fb34b379e298a50c7d264ba"
format_stamp: "Formatted at 2026-09-26 22:48:17 on dist-test-slave-3986"
server_key: "f5dd5c8ec5f813e617fe9daa11c1940b"
server_key_iv: "faea63d0e98a9ce48520b17392819b11"
server_key_version: "encryptionkey@0"
I20260926 22:48:17.199182 468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:17.199846 482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:17.200006 468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:17.200102 468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "fbbf83789fb34b379e298a50c7d264ba"
format_stamp: "Formatted at 2026-09-26 22:48:17 on dist-test-slave-3986"
server_key: "f5dd5c8ec5f813e617fe9daa11c1940b"
server_key_iv: "faea63d0e98a9ce48520b17392819b11"
server_key_version: "encryptionkey@0"
I20260926 22:48:17.200181 468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:17.213959 468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:17.214218 468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:17.214358 468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:17.214558 468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:17.214911 468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:17.214967 468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:17.215006 468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:17.215032 468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:17.220625 468 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:45325
I20260926 22:48:17.220696 595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:45325 every 8 connection(s)
I20260926 22:48:17.221019 468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:48:17.225689 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.221689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:46697 (local address 127.14.184.254:45881)
0926 22:48:17.221788 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.221790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.221986 (+ 196us) server_negotiation.cc:407] Connection header received
0926 22:48:17.222423 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.222424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.222434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.222485 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:17.223198 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.223669 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.224563 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.224748 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.224797 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.225486 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:17.225487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:17.225487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:17.225491 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:17.225504 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:17.225507 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:17.225549 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.225612 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.225632 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 22:48:17.225924 598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.221672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:45881 (local address 127.14.184.195:46697)
0926 22:48:17.221870 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.221902 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.222344 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.222533 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.222535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.222543 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:17.223125 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.223130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.223742 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.223743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.224492 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.224494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.224817 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.224818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.224919 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.225361 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:17.225361 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:17.225387 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:17.225407 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:17.225548 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:17.225550 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:17.225552 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.225596 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.225665 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260926 22:48:17.226390 596 heartbeater.cc:347] Connected to a master server at 127.14.184.254:45881
I20260926 22:48:17.226483 596 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:17.226629 596 heartbeater.cc:511] Master 127.14.184.254:45881 requested a full tablet report, sending...
I20260926 22:48:17.227025 32614 ts_manager.cc:194] Registered new tserver with Master: fbbf83789fb34b379e298a50c7d264ba (127.14.184.195:45325)
I20260926 22:48:17.227351 32614 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.195:46697
I20260926 22:48:17.229526 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 468
I20260926 22:48:17.229614 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:48:17.229769 15075 external_mini_cluster.cc:1428] Setting key dff776a4efd239cc3dd4b7803bebbe21
I20260926 22:48:17.230644 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:48:17.234886 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.231981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59932 (local address 127.14.184.254:45881)
0926 22:48:17.232059 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.232061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.232152 (+ 91us) server_negotiation.cc:407] Connection header received
0926 22:48:17.232220 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.232221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.232230 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.232284 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:17.232459 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.232921 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.233691 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.233886 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.233969 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.234545 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:17.234547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:17.234547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:17.234551 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:17.234564 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:17.234568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:17.234618 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.234783 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.234804 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 22:48:17.236321 32614 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:48:17.237870 32614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:48:17.243875 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.239990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.14.184.195:45325)
0926 22:48:17.240163 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.240167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.240177 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:17.240216 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.240217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.240231 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.240317 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:17.240485 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.241521 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.242716 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.243424 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.243465 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.243552 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.243758 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.243795 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260926 22:48:17.243865 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.239919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:45325 (local address 127.0.0.1:43168)
0926 22:48:17.240012 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.240042 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.240109 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.240328 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.240331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.240334 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:17.240392 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.240394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.241590 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.241591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.242604 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.242607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.243599 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.243600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.243653 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.243661 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.243744 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.243795 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":47,"threads_started":1}
I20260926 22:48:17.244341 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.240270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43809 (local address 127.0.0.1:47390)
0926 22:48:17.240683 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.240692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.240734 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.241072 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.241073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.241075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:17.241113 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.241115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.242304 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.242305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.243198 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.243201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.244100 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.244101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.244162 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.244166 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.244233 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.244275 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":72,"threads_started":1}
I20260926 22:48:17.244354 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.240377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.14.184.193:43809)
0926 22:48:17.240871 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.240875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.240884 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:17.240920 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.240922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.240935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.241062 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:17.241180 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.242242 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.243292 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.244013 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.244087 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.244165 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.244246 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.244282 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":104,"threads_started":1}
I20260926 22:48:17.244642 530 tablet_service.cc:1467] Processing CreateTablet for tablet 42936e10c9954ecc80984c783270b7cf (DEFAULT_TABLE table=test-table [id=4a48af84fef94c0da309496c5d979cda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:17.245007 530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42936e10c9954ecc80984c783270b7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:17.245008 32736 tablet_service.cc:1467] Processing CreateTablet for tablet 42936e10c9954ecc80984c783270b7cf (DEFAULT_TABLE table=test-table [id=4a48af84fef94c0da309496c5d979cda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:17.245348 32736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42936e10c9954ecc80984c783270b7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:17.247529 610 tablet_bootstrap.cc:492] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Bootstrap starting.
I20260926 22:48:17.247944 611 tablet_bootstrap.cc:492] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: Bootstrap starting.
I20260926 22:48:17.248015 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.240105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:36615 (local address 127.0.0.1:54580)
0926 22:48:17.240483 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.240493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.240535 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.240892 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.240893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.240895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:17.240937 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.240939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.242216 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.242217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.243273 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.243276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.247788 (+ 4512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.247789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.247856 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.247860 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.247912 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.247952 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":57,"threads_started":1}
I20260926 22:48:17.248303 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.240480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.14.184.194:36615)
0926 22:48:17.240640 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.240645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.240656 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:48:17.240696 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.240699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.240715 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.240830 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:17.241016 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.242125 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.243349 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.247589 (+ 4240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.247657 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.247754 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.248188 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.248230 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260926 22:48:17.248484 610 tablet_bootstrap.cc:654] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:17.248597 611 tablet_bootstrap.cc:654] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:17.248752 610 log.cc:824] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:17.248884 611 log.cc:824] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:17.248936 399 tablet_service.cc:1467] Processing CreateTablet for tablet 42936e10c9954ecc80984c783270b7cf (DEFAULT_TABLE table=test-table [id=4a48af84fef94c0da309496c5d979cda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:17.249238 399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42936e10c9954ecc80984c783270b7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:17.249421 610 tablet_bootstrap.cc:492] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: No bootstrap required, opened a new log
I20260926 22:48:17.249464 610 ts_tablet_manager.cc:1397] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:17.249624 611 tablet_bootstrap.cc:492] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: No bootstrap required, opened a new log
I20260926 22:48:17.249681 611 ts_tablet_manager.cc:1397] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:17.251473 610 raft_consensus.cc:348] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.251619 610 raft_consensus.cc:374] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:17.251662 610 raft_consensus.cc:729] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbbf83789fb34b379e298a50c7d264ba, State: Initialized, Role: FOLLOWER
I20260926 22:48:17.251777 610 consensus_queue.cc:260] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.252053 610 ts_tablet_manager.cc:1428] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:48:17.252063 596 heartbeater.cc:503] Master 127.14.184.254:45881 was elected leader, sending a full tablet report...
I20260926 22:48:17.252221 611 raft_consensus.cc:348] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.252389 611 raft_consensus.cc:374] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:17.252425 611 raft_consensus.cc:729] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4893943b4847d9987d9f8f4bba9446, State: Initialized, Role: FOLLOWER
I20260926 22:48:17.252555 611 consensus_queue.cc:260] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.252810 611 ts_tablet_manager.cc:1428] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:48:17.252842 334 heartbeater.cc:503] Master 127.14.184.254:45881 was elected leader, sending a full tablet report...
I20260926 22:48:17.252935 615 tablet_bootstrap.cc:492] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Bootstrap starting.
I20260926 22:48:17.253526 615 tablet_bootstrap.cc:654] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:17.253803 615 log.cc:824] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:17.254395 615 tablet_bootstrap.cc:492] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: No bootstrap required, opened a new log
I20260926 22:48:17.254468 615 ts_tablet_manager.cc:1397] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:17.256384 615 raft_consensus.cc:348] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.256537 615 raft_consensus.cc:374] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:17.256582 615 raft_consensus.cc:729] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefe81b965eb430aa1fc01586d0a5a42, State: Initialized, Role: FOLLOWER
I20260926 22:48:17.256690 615 consensus_queue.cc:260] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.256901 615 ts_tablet_manager.cc:1428] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:48:17.256953 465 heartbeater.cc:503] Master 127.14.184.254:45881 was elected leader, sending a full tablet report...
W20260926 22:48:17.352677 466 tablet.cc:2369] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:48:17.367216 618 raft_consensus.cc:482] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:17.367322 618 raft_consensus.cc:504] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.367604 618 leader_election.cc:288] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbbf83789fb34b379e298a50c7d264ba (127.14.184.195:45325), bd4893943b4847d9987d9f8f4bba9446 (127.14.184.193:43809)
I20260926 22:48:17.370625 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.367809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:45325 (local address 127.14.184.194:40905)
0926 22:48:17.367866 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.367879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.367925 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.368089 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.368090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.368092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:17.368201 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.368203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.368941 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.368942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.369719 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.369721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.370374 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.370375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.370444 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.370447 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.370500 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.370531 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 22:48:17.370639 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.367892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:40905 (local address 127.14.184.195:45325)
0926 22:48:17.367946 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.367948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.367961 (+ 13us) server_negotiation.cc:407] Connection header received
0926 22:48:17.367994 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.367995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.368005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.368077 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:17.368300 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.368865 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.369907 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.370295 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.370352 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.370358 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.370519 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.370571 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 22:48:17.370740 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.368030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:58043 (local address 127.14.184.193:43809)
0926 22:48:17.368079 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.368081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.368181 (+ 100us) server_negotiation.cc:407] Connection header received
0926 22:48:17.368209 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.368210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.368221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.368279 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:17.368435 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.368932 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.369968 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.370309 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.370397 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.370405 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.370583 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.370621 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 22:48:17.370774 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.367929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43809 (local address 127.14.184.194:58043)
0926 22:48:17.368099 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.368110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:17.368170 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:17.368299 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:17.368300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:17.368301 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:17.368368 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.368370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.369025 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.369026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.369883 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:17.369885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.370418 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.370419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:17.370504 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.370509 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:48:17.370566 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 22:48:17.370601 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":12,"thread_start_us":94,"threads_started":1}
I20260926 22:48:17.371131 32756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42936e10c9954ecc80984c783270b7cf" candidate_uuid: "cefe81b965eb430aa1fc01586d0a5a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4893943b4847d9987d9f8f4bba9446" is_pre_election: true
I20260926 22:48:17.371129 550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42936e10c9954ecc80984c783270b7cf" candidate_uuid: "cefe81b965eb430aa1fc01586d0a5a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbbf83789fb34b379e298a50c7d264ba" is_pre_election: true
I20260926 22:48:17.371286 550 raft_consensus.cc:2454] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cefe81b965eb430aa1fc01586d0a5a42 in term 0.
I20260926 22:48:17.371286 32756 raft_consensus.cc:2454] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cefe81b965eb430aa1fc01586d0a5a42 in term 0.
I20260926 22:48:17.371515 353 leader_election.cc:302] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4893943b4847d9987d9f8f4bba9446, cefe81b965eb430aa1fc01586d0a5a42; no voters:
I20260926 22:48:17.371637 618 raft_consensus.cc:2781] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:48:17.371692 618 raft_consensus.cc:482] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:17.371721 618 raft_consensus.cc:3037] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:17.372380 618 raft_consensus.cc:504] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.372506 618 leader_election.cc:288] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [CANDIDATE]: Term 1 election: Requested vote from peers fbbf83789fb34b379e298a50c7d264ba (127.14.184.195:45325), bd4893943b4847d9987d9f8f4bba9446 (127.14.184.193:43809)
I20260926 22:48:17.372699 550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42936e10c9954ecc80984c783270b7cf" candidate_uuid: "cefe81b965eb430aa1fc01586d0a5a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbbf83789fb34b379e298a50c7d264ba"
I20260926 22:48:17.372720 32756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42936e10c9954ecc80984c783270b7cf" candidate_uuid: "cefe81b965eb430aa1fc01586d0a5a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4893943b4847d9987d9f8f4bba9446"
I20260926 22:48:17.372783 550 raft_consensus.cc:3037] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:17.372792 32756 raft_consensus.cc:3037] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:17.373692 550 raft_consensus.cc:2454] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cefe81b965eb430aa1fc01586d0a5a42 in term 1.
I20260926 22:48:17.373692 32756 raft_consensus.cc:2454] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cefe81b965eb430aa1fc01586d0a5a42 in term 1.
I20260926 22:48:17.373879 353 leader_election.cc:302] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cefe81b965eb430aa1fc01586d0a5a42, fbbf83789fb34b379e298a50c7d264ba; no voters:
I20260926 22:48:17.373973 618 raft_consensus.cc:2781] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:17.374109 618 raft_consensus.cc:686] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 1 LEADER]: Becoming Leader. State: Replica: cefe81b965eb430aa1fc01586d0a5a42, State: Running, Role: LEADER
I20260926 22:48:17.374328 618 consensus_queue.cc:237] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } }
I20260926 22:48:17.375161 32614 catalog_manager.cc:5495] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 reported cstate change: term changed from 0 to 1, leader changed from <none> to cefe81b965eb430aa1fc01586d0a5a42 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "cefe81b965eb430aa1fc01586d0a5a42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cefe81b965eb430aa1fc01586d0a5a42" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 36615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 } health_report { overall_health: UNKNOWN } } }
I20260926 22:48:17.387267 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:17.385116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.14.184.194:36615)
0926 22:48:17.385155 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:17.385157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:17.385176 (+ 19us) server_negotiation.cc:407] Connection header received
0926 22:48:17.385224 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:17.385225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:17.385239 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:17.385296 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:17.385397 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.385863 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.386567 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:17.386792 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:17.386845 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:17.386968 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:17.387055 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:17.387108 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:17.387182 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:48:17.387204 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:48:17.388906 550 raft_consensus.cc:1261] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 1 FOLLOWER]: Refusing update from remote peer cefe81b965eb430aa1fc01586d0a5a42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:17.388934 32756 raft_consensus.cc:1261] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 1 FOLLOWER]: Refusing update from remote peer cefe81b965eb430aa1fc01586d0a5a42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:17.389289 620 consensus_queue.cc:1035] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4893943b4847d9987d9f8f4bba9446" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 43809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:17.389427 618 consensus_queue.cc:1035] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbbf83789fb34b379e298a50c7d264ba" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 45325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:17.391855 624 mvcc.cc:205] Tried to move back new op lower bound from 7333736027702431744 to 7333736027645943808. Current Snapshot: MvccSnapshot[applied={T|T < 7333736027702431744}]
I20260926 22:48:17.391940 625 mvcc.cc:205] Tried to move back new op lower bound from 7333736027702431744 to 7333736027645943808. Current Snapshot: MvccSnapshot[applied={T|T < 7333736027702431744}]
I20260926 22:48:17.394083 32613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:17.394177 32613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:17.394934 32613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77138a04c4ae4f63a73c06f72e76f31b: Sending DeleteTablet for 3 replicas of tablet 42936e10c9954ecc80984c783270b7cf
I20260926 22:48:17.395251 530 tablet_service.cc:1514] Processing DeleteTablet for tablet 42936e10c9954ecc80984c783270b7cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:17 UTC) from {username='slave'} at 127.0.0.1:43168
I20260926 22:48:17.395308 399 tablet_service.cc:1514] Processing DeleteTablet for tablet 42936e10c9954ecc80984c783270b7cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:17 UTC) from {username='slave'} at 127.0.0.1:54580
I20260926 22:48:17.395320 32736 tablet_service.cc:1514] Processing DeleteTablet for tablet 42936e10c9954ecc80984c783270b7cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:17 UTC) from {username='slave'} at 127.0.0.1:47390
I20260926 22:48:17.395517 634 tablet_replica.cc:330] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: stopping tablet replica
I20260926 22:48:17.395531 633 tablet_replica.cc:330] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: stopping tablet replica
I20260926 22:48:17.395618 633 raft_consensus.cc:2229] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:17.395720 633 raft_consensus.cc:2258] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:17.395716 635 tablet_replica.cc:330] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446: stopping tablet replica
I20260926 22:48:17.395747 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 32674
I20260926 22:48:17.395876 635 raft_consensus.cc:2229] T 42936e10c9954ecc80984c783270b7cf P bd4893943b4847d9987d9f8f4bba9446 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:17.395979 634 raft_consensus.cc:2229] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:48:17.396096 634 raft_consensus.cc:2258] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:17.396790 634 ts_tablet_manager.cc:1905] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:17.396942 633 ts_tablet_manager.cc:1905] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:17.398492 634 ts_tablet_manager.cc:1918] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:17.398624 634 log.cc:1196] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/42936e10c9954ecc80984c783270b7cf
I20260926 22:48:17.398880 633 ts_tablet_manager.cc:1918] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:17.398952 633 log.cc:1196] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/42936e10c9954ecc80984c783270b7cf
I20260926 22:48:17.399075 634 ts_tablet_manager.cc:1939] T 42936e10c9954ecc80984c783270b7cf P cefe81b965eb430aa1fc01586d0a5a42: Deleting consensus metadata
I20260926 22:48:17.399205 633 ts_tablet_manager.cc:1939] T 42936e10c9954ecc80984c783270b7cf P fbbf83789fb34b379e298a50c7d264ba: Deleting consensus metadata
I20260926 22:48:17.399580 32599 catalog_manager.cc:4841] TS fbbf83789fb34b379e298a50c7d264ba (127.14.184.195:45325): tablet 42936e10c9954ecc80984c783270b7cf (table test-table [id=4a48af84fef94c0da309496c5d979cda]) successfully deleted
W20260926 22:48:17.402101 32599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:43809 (error 108)
I20260926 22:48:17.402531 32599 catalog_manager.cc:4841] TS cefe81b965eb430aa1fc01586d0a5a42 (127.14.184.194:36615): tablet 42936e10c9954ecc80984c783270b7cf (table test-table [id=4a48af84fef94c0da309496c5d979cda]) successfully deleted
W20260926 22:48:17.402659 606 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:17.402479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:43809 (local address 127.0.0.1:59380)
0926 22:48:17.402546 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:17.402598 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:43809: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 22:48:17.402796 32599 catalog_manager.cc:4584] TS bd4893943b4847d9987d9f8f4bba9446 (127.14.184.193:43809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42936e10c9954ecc80984c783270b7cf: Network error: Client connection negotiation failed: client connection to 127.14.184.193:43809: connect: Connection reset by peer (error 104)
I20260926 22:48:17.402853 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 337
I20260926 22:48:17.408385 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 468
I20260926 22:48:17.412686 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 32583
2026-09-26T22:48:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (717 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:17 dist-test-slave-3986 krb5kdc[639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:17 dist-test-slave-3986 krb5kdc[639](info): set up 2 sockets
Sep 26 22:48:17 dist-test-slave-3986 krb5kdc[639](info): commencing operation
krb5kdc: starting...
W20260926 22:48:18.438736 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:18 dist-test-slave-3986 krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:18.555480 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:41337
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:41337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:18.632910 652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:18.633129 652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:18.633165 652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:18.634282 652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:18.634346 652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:18.634369 652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:18.634392 652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:18.634418 652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:18.635874 652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:41337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:41337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:18.636137 652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:18.636348 652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:18.641104 657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:18.641076 658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:18.641083 660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:18.641561 652 server_base.cc:1034] running on GCE node
I20260926 22:48:18.641837 652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:18.642061 652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:18.643193 652 hybrid_clock.cc:648] HybridClock initialized: now 1790462898643176 us; error 28 us; skew 500 ppm
Sep 26 22:48:18 dist-test-slave-3986 krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462898, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:18.645762 652 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:18.646127 652 webserver.cc:460] Webserver started at http://127.14.184.254:37249/ using document root <none> and password file <none>
I20260926 22:48:18.646329 652 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:18.646368 652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:18.646451 652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:18.647519 652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "c7e915f30ff041c49bc87c401016d7e9"
format_stamp: "Formatted at 2026-09-26 22:48:18 on dist-test-slave-3986"
server_key: "22b2561bba7ea396ce5564e01457e3b2"
server_key_iv: "42135c22feea6966aaab5898a9d84272"
server_key_version: "encryptionkey@0"
I20260926 22:48:18.647806 652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "c7e915f30ff041c49bc87c401016d7e9"
format_stamp: "Formatted at 2026-09-26 22:48:18 on dist-test-slave-3986"
server_key: "22b2561bba7ea396ce5564e01457e3b2"
server_key_iv: "42135c22feea6966aaab5898a9d84272"
server_key_version: "encryptionkey@0"
I20260926 22:48:18.649004 652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:48:18.649724 667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:18.649873 652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:18.649958 652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "c7e915f30ff041c49bc87c401016d7e9"
format_stamp: "Formatted at 2026-09-26 22:48:18 on dist-test-slave-3986"
server_key: "22b2561bba7ea396ce5564e01457e3b2"
server_key_iv: "42135c22feea6966aaab5898a9d84272"
server_key_version: "encryptionkey@0"
I20260926 22:48:18.650012 652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:18.658629 652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:18.660742 652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:18.660841 652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:18.664551 652 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:41337
I20260926 22:48:18.664603 719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:41337 every 8 connection(s)
I20260926 22:48:18.664924 652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:18.665747 720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:18.667994 720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: Bootstrap starting.
I20260926 22:48:18.668545 720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:18.668836 720 log.cc:824] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:18.669507 720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: No bootstrap required, opened a new log
I20260926 22:48:18.669528 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 652
I20260926 22:48:18.669634 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:18.669835 15075 external_mini_cluster.cc:1428] Setting key 08987c31905489bce47f4eca3e7dc998
I20260926 22:48:18.671767 720 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e915f30ff041c49bc87c401016d7e9" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41337 } }
I20260926 22:48:18.671939 720 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:18.671981 720 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e915f30ff041c49bc87c401016d7e9, State: Initialized, Role: FOLLOWER
I20260926 22:48:18.672124 720 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e915f30ff041c49bc87c401016d7e9" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41337 } }
I20260926 22:48:18.672200 720 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:18.672230 720 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:18.672264 720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:18.672907 720 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e915f30ff041c49bc87c401016d7e9" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41337 } }
I20260926 22:48:18.673009 720 leader_election.cc:302] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7e915f30ff041c49bc87c401016d7e9; no voters:
I20260926 22:48:18.673147 720 leader_election.cc:288] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:18.673197 725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:18.673362 720 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:18.673372 725 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [term 1 LEADER]: Becoming Leader. State: Replica: c7e915f30ff041c49bc87c401016d7e9, State: Running, Role: LEADER
I20260926 22:48:18.673522 725 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e915f30ff041c49bc87c401016d7e9" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41337 } }
I20260926 22:48:18.673849 725 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e915f30ff041c49bc87c401016d7e9. Latest consensus state: current_term: 1 leader_uuid: "c7e915f30ff041c49bc87c401016d7e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e915f30ff041c49bc87c401016d7e9" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41337 } } }
I20260926 22:48:18.673990 725 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:18.674754 726 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7e915f30ff041c49bc87c401016d7e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e915f30ff041c49bc87c401016d7e9" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41337 } } }
I20260926 22:48:18.675091 726 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9 [sys.catalog]: This master's current role is: LEADER
W20260926 22:48:18.675112 741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:18.675145 741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:18.675221 734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:18.675473 734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:18.676877 734 catalog_manager.cc:1347] Generated new cluster ID: 77a796bdd857417c8ff07a663d3e8943
I20260926 22:48:18.676927 734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:18 dist-test-slave-3986 krb5kdc[639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:18.682269 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:18.670318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.14.184.254:41337)
0926 22:48:18.670479 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:18.670489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:18.670524 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:48:18.670984 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:18.670992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:18.671049 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:18.671191 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:18.671763 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:18.673406 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:18.674477 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:18.675195 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:18.675306 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:18.677975 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:18.677979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:18.677979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:18.677990 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:18.679950 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:18.680360 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:18.680363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:18.680363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:18.680463 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:18.680664 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:18.680665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:18.680665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:18.680972 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:18.681129 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:18.681933 (+ 804us) server_negotiation.cc:299] Negotiation successful
0926 22:48:18.682090 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260926 22:48:18.702049 734 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:18.702594 734 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:18.704325 734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: Generated new TSK 0
I20260926 22:48:18.704484 734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:48:18.737650 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:41337
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:18.850244 746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:18.850452 746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:18.850476 746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:18.851643 746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:18.851686 746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:18.851711 746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:48:18.853127 746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:41337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:18.853309 746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:18.853473 746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:18.858456 754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:18.858465 752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:18.858474 746 server_base.cc:1034] running on GCE node
W20260926 22:48:18.858493 751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:18.858844 746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:18.859086 746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:18.860260 746 hybrid_clock.cc:648] HybridClock initialized: now 1790462898860243 us; error 27 us; skew 500 ppm
Sep 26 22:48:18 dist-test-slave-3986 krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462898, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:18.862636 746 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:48:18.863116 746 webserver.cc:460] Webserver started at http://127.14.184.193:33415/ using document root <none> and password file <none>
I20260926 22:48:18.863339 746 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:18.863407 746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:18.863554 746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:18.864562 746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "2a750f5e6f9649e6b5af9a3acf94583d"
format_stamp: "Formatted at 2026-09-26 22:48:18 on dist-test-slave-3986"
server_key: "f690a44ba3f162da2e7a7ef4d8eef49d"
server_key_iv: "9314d001c7cae3b3c19fec3dcd1be45f"
server_key_version: "encryptionkey@0"
I20260926 22:48:18.864916 746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "2a750f5e6f9649e6b5af9a3acf94583d"
format_stamp: "Formatted at 2026-09-26 22:48:18 on dist-test-slave-3986"
server_key: "f690a44ba3f162da2e7a7ef4d8eef49d"
server_key_iv: "9314d001c7cae3b3c19fec3dcd1be45f"
server_key_version: "encryptionkey@0"
I20260926 22:48:18.866205 746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:18.866958 761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:18.867126 746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:18.867252 746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "2a750f5e6f9649e6b5af9a3acf94583d"
format_stamp: "Formatted at 2026-09-26 22:48:18 on dist-test-slave-3986"
server_key: "f690a44ba3f162da2e7a7ef4d8eef49d"
server_key_iv: "9314d001c7cae3b3c19fec3dcd1be45f"
server_key_version: "encryptionkey@0"
I20260926 22:48:18.867345 746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:18.889282 746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:18.891278 746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:18.891438 746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:18.891671 746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:18.891992 746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:18.892046 746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:18.892088 746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:18.892117 746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:18.897895 746 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:42251
I20260926 22:48:18.897945 874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:42251 every 8 connection(s)
I20260926 22:48:18.898247 746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:48:18.902855 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 746
I20260926 22:48:18.902976 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:48:18.903165 15075 external_mini_cluster.cc:1428] Setting key dcba8e6189db48f0045054def2c4deb7
Sep 26 22:48:18 dist-test-slave-3986 krb5kdc[639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462898, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:18.907966 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:18.898932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:48171 (local address 127.14.184.254:41337)
0926 22:48:18.898979 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:18.898981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:18.899185 (+ 204us) server_negotiation.cc:407] Connection header received
0926 22:48:18.899736 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:18.899738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:18.899750 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:18.899813 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:18.900764 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:18.901264 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:18.902281 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:18.902500 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:18.902572 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:18.905100 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:18.905102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:18.905102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:18.905110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:18.906473 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:18.906991 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:18.906992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:18.906992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:18.907043 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:18.907335 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:18.907335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:18.907336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:18.907434 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:18.907485 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:18.907832 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 22:48:18.907903 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:18.908241 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:18.898909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:41337 (local address 127.14.184.193:48171)
0926 22:48:18.899059 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:18.899102 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:18.899616 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:18.899885 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:18.899889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:18.900449 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:18.900641 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:18.900646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:18.901363 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:18.901364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:18.902178 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:18.902180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:18.902578 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:18.902580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:18.902790 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:18.903288 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:18.903299 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:18.905007 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:18.906556 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:18.906563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:18.906565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:18.906911 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:18.907113 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:18.907114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:18.907115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:18.907268 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:18.907502 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:18.907510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:18.907682 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 22:48:18.907810 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 22:48:18.907986 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260926 22:48:18.908818 875 heartbeater.cc:347] Connected to a master server at 127.14.184.254:41337
I20260926 22:48:18.908910 875 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:18.909097 875 heartbeater.cc:511] Master 127.14.184.254:41337 requested a full tablet report, sending...
I20260926 22:48:18.909595 684 ts_manager.cc:194] Registered new tserver with Master: 2a750f5e6f9649e6b5af9a3acf94583d (127.14.184.193:42251)
I20260926 22:48:18.910138 684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:48171
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:48:18.927474 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:41337
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:19.005604 880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:19.005831 880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:19.005869 880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:19.006954 880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:19.007017 880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:19.007061 880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:48:19.008451 880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:41337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:19.008711 880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:19.008934 880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:19.013690 886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.013691 888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.013695 885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:19.013763 880 server_base.cc:1034] running on GCE node
I20260926 22:48:19.014124 880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:19.014353 880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:19.015489 880 hybrid_clock.cc:648] HybridClock initialized: now 1790462899015478 us; error 27 us; skew 500 ppm
Sep 26 22:48:19 dist-test-slave-3986 krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462899, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:19.017917 880 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:48:19.018358 880 webserver.cc:460] Webserver started at http://127.14.184.194:39505/ using document root <none> and password file <none>
I20260926 22:48:19.018580 880 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:19.018623 880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:19.018805 880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:19.019770 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "759f50bb55626225558f630770d8f7b5"
server_key_iv: "95ee79e740edbeae3abb921695d3b531"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.020119 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "759f50bb55626225558f630770d8f7b5"
server_key_iv: "95ee79e740edbeae3abb921695d3b531"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.021363 880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:48:19.022059 895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.022222 880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:19.022315 880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "759f50bb55626225558f630770d8f7b5"
server_key_iv: "95ee79e740edbeae3abb921695d3b531"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.022392 880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:19.039075 880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:19.041241 880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:19.041415 880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:19.041620 880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:19.041919 880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:19.041978 880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.042018 880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:19.042047 880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.047629 880 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:41597
I20260926 22:48:19.047676 1008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:41597 every 8 connection(s)
I20260926 22:48:19.047962 880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:48:19.051714 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 880
I20260926 22:48:19.051828 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:48:19.052047 15075 external_mini_cluster.cc:1428] Setting key 5fb57a917f48480f7fa5492d5af2dd9f
Sep 26 22:48:19 dist-test-slave-3986 krb5kdc[639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462899, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:19.057245 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.048589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:34169 (local address 127.14.184.254:41337)
0926 22:48:19.048648 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.048650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.048839 (+ 189us) server_negotiation.cc:407] Connection header received
0926 22:48:19.049298 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.049299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.049308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.049363 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:19.050192 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.050713 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.051638 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.051937 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.052013 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.054101 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:19.054102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:19.054103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:19.054111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:19.055740 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:19.056254 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:19.056255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:19.056258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:19.056310 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:19.056570 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:19.056571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:19.056571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:19.056722 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:19.056803 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.057076 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.057154 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 22:48:19.057449 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.048564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:41337 (local address 127.14.184.194:34169)
0926 22:48:19.048723 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.048762 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.049220 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.049414 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.049417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.049947 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:19.050120 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.050124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.050805 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.050805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.051562 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.051564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.052019 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.052020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.052162 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.052665 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:19.052674 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:19.053996 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:19.055832 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:19.055839 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:19.055841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:19.056165 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:19.056389 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:19.056390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:19.056391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:19.056500 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:19.056793 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:19.056800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:19.056938 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.057042 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.057212 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260926 22:48:19.057986 1009 heartbeater.cc:347] Connected to a master server at 127.14.184.254:41337
I20260926 22:48:19.058104 1009 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:19.058302 1009 heartbeater.cc:511] Master 127.14.184.254:41337 requested a full tablet report, sending...
I20260926 22:48:19.058662 684 ts_manager.cc:194] Registered new tserver with Master: ad72461eebf2467b9dd9a7fb03e1ee1c (127.14.184.194:41597)
I20260926 22:48:19.059105 684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:34169
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:48:19.075647 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:41337
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:35195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:19.155918 1014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:19.156121 1014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:19.156147 1014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:19.157212 1014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:19.157255 1014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:19.157281 1014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:48:19.158792 1014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:35195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:41337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:19.159042 1014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:19.159245 1014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:19.164247 1022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.164255 1020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.164324 1019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:19.164592 1014 server_base.cc:1034] running on GCE node
I20260926 22:48:19.164805 1014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:19.165071 1014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:19.166247 1014 hybrid_clock.cc:648] HybridClock initialized: now 1790462899166237 us; error 39 us; skew 500 ppm
Sep 26 22:48:19 dist-test-slave-3986 krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462899, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:19.168833 1014 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:48:19.169237 1014 webserver.cc:460] Webserver started at http://127.14.184.195:35125/ using document root <none> and password file <none>
I20260926 22:48:19.169469 1014 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:19.169539 1014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:19.169696 1014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:19.171064 1014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "f14fb78c84454f318504b3d5c8748699"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "8043cc85b7f320f48141810d21e3f697"
server_key_iv: "4d5a1f29877de777b4099829e090e8cf"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.171439 1014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "f14fb78c84454f318504b3d5c8748699"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "8043cc85b7f320f48141810d21e3f697"
server_key_iv: "4d5a1f29877de777b4099829e090e8cf"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.172737 1014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:19.173609 1029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.173846 1014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:19.173949 1014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "f14fb78c84454f318504b3d5c8748699"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "8043cc85b7f320f48141810d21e3f697"
server_key_iv: "4d5a1f29877de777b4099829e090e8cf"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.174057 1014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:19.189469 1014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:19.192638 1014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:19.192816 1014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:19.193087 1014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:19.193464 1014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:19.193519 1014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.193557 1014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:19.193589 1014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.201268 1014 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:39513
I20260926 22:48:19.201298 1142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:39513 every 8 connection(s)
I20260926 22:48:19.201745 1014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:48:19.202200 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1014
I20260926 22:48:19.202301 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:48:19.202468 15075 external_mini_cluster.cc:1428] Setting key aa69e6af9dd90adeab6bab270bc9dcbd
Sep 26 22:48:19 dist-test-slave-3986 krb5kdc[639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462899, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:19.212765 1145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.203158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:41337 (local address 127.14.184.195:34355)
0926 22:48:19.203330 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.203369 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.203892 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.204153 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.204157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.204764 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:19.204949 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.204953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.206071 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.206071 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.206935 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.206938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.207463 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.207464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.207557 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.208060 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:19.208068 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:19.209636 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:19.211527 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:19.211532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:19.211534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:19.211752 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:19.211955 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:19.211955 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:19.211955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:19.212044 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:19.212331 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:19.212334 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:19.212432 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.212503 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.212606 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260926 22:48:19.212785 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.203111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:34355 (local address 127.14.184.254:41337)
0926 22:48:19.203158 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.203160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.203436 (+ 276us) server_negotiation.cc:407] Connection header received
0926 22:48:19.203987 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.203988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.204001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.204078 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:19.205050 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.205969 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.207011 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.207380 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.207461 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.209733 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:19.209736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:19.209736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:19.209744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:19.211449 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:19.211824 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:19.211825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:19.211826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:19.211889 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:19.212104 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:19.212105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:19.212106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:19.212260 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:19.212333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.212560 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.212648 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:48:19.213340 1143 heartbeater.cc:347] Connected to a master server at 127.14.184.254:41337
I20260926 22:48:19.213439 1143 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:19.213650 1143 heartbeater.cc:511] Master 127.14.184.254:41337 requested a full tablet report, sending...
I20260926 22:48:19.214022 684 ts_manager.cc:194] Registered new tserver with Master: f14fb78c84454f318504b3d5c8748699 (127.14.184.195:39513)
I20260926 22:48:19.214242 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:48:19.214567 684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:34355
I20260926 22:48:19.221791 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.215781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.14.184.254:41337)
0926 22:48:19.215828 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.215830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.215941 (+ 111us) server_negotiation.cc:407] Connection header received
0926 22:48:19.216015 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.216015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.216025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.216078 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:19.216618 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.217064 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.217775 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.217977 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.218059 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.219325 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:19.219326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:19.219327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:19.219333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:19.220606 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:19.220990 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:19.220991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:19.220992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:19.221045 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:19.221265 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:19.221266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:19.221266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:19.221387 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:19.221497 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.221626 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.221689 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 22:48:19.223465 684 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:48:19.225157 684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:48:19.232184 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.227372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:41597 (local address 127.0.0.1:47470)
0926 22:48:19.227831 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.227867 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.227938 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.228207 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.228210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.228212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:19.228274 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.228275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.229579 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.229580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.230949 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.230951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.231911 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.231992 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.231996 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.232059 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.232116 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":63,"thread_start_us":90,"threads_started":1}
I20260926 22:48:19.232203 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.227567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.14.184.195:39513)
0926 22:48:19.227749 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.227754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.228131 (+ 377us) server_negotiation.cc:407] Connection header received
0926 22:48:19.228167 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.228169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.228184 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.228261 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:19.228422 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.229461 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231046 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.231737 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231808 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.231864 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.232100 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.232137 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
I20260926 22:48:19.232546 1158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.227570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.14.184.194:41597)
0926 22:48:19.227749 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.227754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.227947 (+ 193us) server_negotiation.cc:407] Connection header received
0926 22:48:19.228017 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.228020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.228038 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.228114 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:19.228351 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.229461 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231045 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.231785 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231840 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.231884 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.232421 (+ 537us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.232471 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260926 22:48:19.232822 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.227372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:39513 (local address 127.0.0.1:45916)
0926 22:48:19.228022 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.228031 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.228074 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.228315 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.228316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.228318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:19.228350 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.228350 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.229580 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.229581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.230948 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.230951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.231899 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.231982 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.231994 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.232059 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.232751 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":690,"thread_start_us":85,"threads_started":1}
I20260926 22:48:19.233184 943 tablet_service.cc:1467] Processing CreateTablet for tablet 88b2d11fdd9547a994a8a85594e767c4 (DEFAULT_TABLE table=test-table [id=ddaa0c9b40dc4bf68960d1b51d639a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:19.233469 1156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.227370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:42251 (local address 127.0.0.1:41400)
0926 22:48:19.227806 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.227847 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.227905 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.228643 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.228644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.228646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:19.228706 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.228708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.230450 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.230451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.231827 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.231830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.233251 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.233252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.233312 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.233316 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.233367 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.233405 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":53,"threads_started":1}
I20260926 22:48:19.233577 943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88b2d11fdd9547a994a8a85594e767c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:19.233451 1077 tablet_service.cc:1467] Processing CreateTablet for tablet 88b2d11fdd9547a994a8a85594e767c4 (DEFAULT_TABLE table=test-table [id=ddaa0c9b40dc4bf68960d1b51d639a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:19.233789 1077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88b2d11fdd9547a994a8a85594e767c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:19.233806 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.227620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.14.184.193:42251)
0926 22:48:19.228070 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.228074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.228085 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:48:19.228126 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.228129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.228145 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.228549 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:19.228789 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.230027 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.231933 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.233093 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.233163 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.233213 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.233631 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.233673 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":121,"threads_started":1}
I20260926 22:48:19.234354 809 tablet_service.cc:1467] Processing CreateTablet for tablet 88b2d11fdd9547a994a8a85594e767c4 (DEFAULT_TABLE table=test-table [id=ddaa0c9b40dc4bf68960d1b51d639a1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:19.234711 809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88b2d11fdd9547a994a8a85594e767c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:19.236402 1160 tablet_bootstrap.cc:492] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Bootstrap starting.
I20260926 22:48:19.236910 1161 tablet_bootstrap.cc:492] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Bootstrap starting.
I20260926 22:48:19.237016 1160 tablet_bootstrap.cc:654] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:19.237612 1160 log.cc:824] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:19.238142 1161 tablet_bootstrap.cc:654] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:19.238370 1160 tablet_bootstrap.cc:492] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: No bootstrap required, opened a new log
I20260926 22:48:19.238442 1160 ts_tablet_manager.cc:1397] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:19.238559 1163 tablet_bootstrap.cc:492] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: Bootstrap starting.
I20260926 22:48:19.238617 1161 log.cc:824] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:19.239250 1163 tablet_bootstrap.cc:654] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:19.239845 1163 log.cc:824] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:19.240101 1160 raft_consensus.cc:348] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.240305 1160 raft_consensus.cc:374] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:19.240355 1160 raft_consensus.cc:729] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14fb78c84454f318504b3d5c8748699, State: Initialized, Role: FOLLOWER
I20260926 22:48:19.240468 1160 consensus_queue.cc:260] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.240744 1160 ts_tablet_manager.cc:1428] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:48:19.240828 1163 tablet_bootstrap.cc:492] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: No bootstrap required, opened a new log
I20260926 22:48:19.240877 1143 heartbeater.cc:503] Master 127.14.184.254:41337 was elected leader, sending a full tablet report...
I20260926 22:48:19.240904 1163 ts_tablet_manager.cc:1397] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:19.241637 1161 tablet_bootstrap.cc:492] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: No bootstrap required, opened a new log
I20260926 22:48:19.241701 1161 ts_tablet_manager.cc:1397] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 22:48:19.243294 1161 raft_consensus.cc:348] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.243448 1161 raft_consensus.cc:374] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:19.243489 1161 raft_consensus.cc:729] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad72461eebf2467b9dd9a7fb03e1ee1c, State: Initialized, Role: FOLLOWER
I20260926 22:48:19.243604 1161 consensus_queue.cc:260] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.243885 1009 heartbeater.cc:503] Master 127.14.184.254:41337 was elected leader, sending a full tablet report...
I20260926 22:48:19.243899 1161 ts_tablet_manager.cc:1428] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 22:48:19.244002 1163 raft_consensus.cc:348] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.244199 1163 raft_consensus.cc:374] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:19.244246 1163 raft_consensus.cc:729] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a750f5e6f9649e6b5af9a3acf94583d, State: Initialized, Role: FOLLOWER
I20260926 22:48:19.244369 1163 consensus_queue.cc:260] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.244670 875 heartbeater.cc:503] Master 127.14.184.254:41337 was elected leader, sending a full tablet report...
I20260926 22:48:19.244664 1163 ts_tablet_manager.cc:1428] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260926 22:48:19.298480 1010 tablet.cc:2369] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:48:19.398991 876 tablet.cc:2369] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 22:48:19.452327 1144 tablet.cc:2369] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:48:19.588896 1168 raft_consensus.cc:482] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:19.589052 1168 raft_consensus.cc:504] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.589445 1168 leader_election.cc:288] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f14fb78c84454f318504b3d5c8748699 (127.14.184.195:39513), ad72461eebf2467b9dd9a7fb03e1ee1c (127.14.184.194:41597)
I20260926 22:48:19.593459 1158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.589727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:40233 (local address 127.14.184.194:41597)
0926 22:48:19.589770 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.589771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.589830 (+ 59us) server_negotiation.cc:407] Connection header received
0926 22:48:19.589906 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.589907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.589917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.589973 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:19.590177 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.590947 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.592532 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.593083 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.593164 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.593173 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.593350 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.593390 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:48:19.593459 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.589680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:59091 (local address 127.14.184.195:39513)
0926 22:48:19.589734 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.589736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.589832 (+ 96us) server_negotiation.cc:407] Connection header received
0926 22:48:19.589907 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.589907 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.589917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.589973 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:19.590176 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.590947 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.592531 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.593091 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.593165 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.593173 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.593350 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.593388 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:48:19.593513 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.589622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:41597 (local address 127.14.184.193:40233)
0926 22:48:19.589772 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.589780 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.589832 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.589997 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.589998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.590002 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:19.590099 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.590100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.591033 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.591034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.592434 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.592437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.593179 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.593180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.593261 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.593267 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.593326 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.593381 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":67,"thread_start_us":65,"threads_started":1}
I20260926 22:48:19.593516 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.589601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:39513 (local address 127.14.184.193:59091)
0926 22:48:19.589739 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.589752 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.589832 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.589997 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.589998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.590000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:19.590095 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.590097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.591034 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.591035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.592435 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.592437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.593181 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.593181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.593261 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.593266 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.593326 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.593381 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":116,"thread_start_us":59,"threads_started":1}
I20260926 22:48:19.593861 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88b2d11fdd9547a994a8a85594e767c4" candidate_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" is_pre_election: true
I20260926 22:48:19.593923 1097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88b2d11fdd9547a994a8a85594e767c4" candidate_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f14fb78c84454f318504b3d5c8748699" is_pre_election: true
I20260926 22:48:19.594027 963 raft_consensus.cc:2454] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a750f5e6f9649e6b5af9a3acf94583d in term 0.
I20260926 22:48:19.594071 1097 raft_consensus.cc:2454] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a750f5e6f9649e6b5af9a3acf94583d in term 0.
I20260926 22:48:19.594277 764 leader_election.cc:302] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a750f5e6f9649e6b5af9a3acf94583d, ad72461eebf2467b9dd9a7fb03e1ee1c; no voters:
I20260926 22:48:19.594406 1168 raft_consensus.cc:2781] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:48:19.594462 1168 raft_consensus.cc:482] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:19.594485 1168 raft_consensus.cc:3037] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:19.595395 1168 raft_consensus.cc:504] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.595543 1168 leader_election.cc:288] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [CANDIDATE]: Term 1 election: Requested vote from peers f14fb78c84454f318504b3d5c8748699 (127.14.184.195:39513), ad72461eebf2467b9dd9a7fb03e1ee1c (127.14.184.194:41597)
I20260926 22:48:19.595715 1097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88b2d11fdd9547a994a8a85594e767c4" candidate_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f14fb78c84454f318504b3d5c8748699"
I20260926 22:48:19.595782 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88b2d11fdd9547a994a8a85594e767c4" candidate_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c"
I20260926 22:48:19.595811 1097 raft_consensus.cc:3037] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:19.595881 963 raft_consensus.cc:3037] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:19.596522 1097 raft_consensus.cc:2454] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a750f5e6f9649e6b5af9a3acf94583d in term 1.
I20260926 22:48:19.596604 963 raft_consensus.cc:2454] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a750f5e6f9649e6b5af9a3acf94583d in term 1.
I20260926 22:48:19.596825 765 leader_election.cc:302] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a750f5e6f9649e6b5af9a3acf94583d, f14fb78c84454f318504b3d5c8748699; no voters:
I20260926 22:48:19.596943 1168 raft_consensus.cc:2781] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:19.597117 1168 raft_consensus.cc:686] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [term 1 LEADER]: Becoming Leader. State: Replica: 2a750f5e6f9649e6b5af9a3acf94583d, State: Running, Role: LEADER
I20260926 22:48:19.597221 1168 consensus_queue.cc:237] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } }
I20260926 22:48:19.598060 684 catalog_manager.cc:5495] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a750f5e6f9649e6b5af9a3acf94583d (127.14.184.193). New cstate: current_term: 1 leader_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a750f5e6f9649e6b5af9a3acf94583d" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 42251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 } health_report { overall_health: UNKNOWN } } }
I20260926 22:48:19.675372 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.672717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.14.184.193:42251)
0926 22:48:19.672789 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.672790 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.672798 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:48:19.672835 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.672836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.672854 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.672915 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:19.673080 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.673751 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.674623 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.674892 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.674953 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.675070 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:19.675157 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:19.675224 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.675244 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.675254 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 22:48:19.677349 963 raft_consensus.cc:1261] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 1 FOLLOWER]: Refusing update from remote peer 2a750f5e6f9649e6b5af9a3acf94583d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:19.677349 1097 raft_consensus.cc:1261] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 1 FOLLOWER]: Refusing update from remote peer 2a750f5e6f9649e6b5af9a3acf94583d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:19.677714 1168 consensus_queue.cc:1035] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad72461eebf2467b9dd9a7fb03e1ee1c" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:19.677898 1171 consensus_queue.cc:1035] T 88b2d11fdd9547a994a8a85594e767c4 P 2a750f5e6f9649e6b5af9a3acf94583d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f14fb78c84454f318504b3d5c8748699" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:19.680212 1174 mvcc.cc:205] Tried to move back new op lower bound from 7333736037075546112 to 7333736036750794752. Current Snapshot: MvccSnapshot[applied={T|T < 7333736037075546112}]
I20260926 22:48:19.680481 1176 mvcc.cc:205] Tried to move back new op lower bound from 7333736037075546112 to 7333736036750794752. Current Snapshot: MvccSnapshot[applied={T|T < 7333736037075546112}]
I20260926 22:48:19.681170 1175 mvcc.cc:205] Tried to move back new op lower bound from 7333736037075546112 to 7333736036750794752. Current Snapshot: MvccSnapshot[applied={T|T < 7333736037075546112 or (T in {7333736037075546112})}]
I20260926 22:48:19.682902 684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:19.683000 684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:19.683779 684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7e915f30ff041c49bc87c401016d7e9: Sending DeleteTablet for 3 replicas of tablet 88b2d11fdd9547a994a8a85594e767c4
I20260926 22:48:19.684163 1077 tablet_service.cc:1514] Processing DeleteTablet for tablet 88b2d11fdd9547a994a8a85594e767c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:19 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:45916
I20260926 22:48:19.684168 943 tablet_service.cc:1514] Processing DeleteTablet for tablet 88b2d11fdd9547a994a8a85594e767c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:19 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:47470
I20260926 22:48:19.684382 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 746
I20260926 22:48:19.684437 1184 tablet_replica.cc:330] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: stopping tablet replica
I20260926 22:48:19.684475 809 tablet_service.cc:1514] Processing DeleteTablet for tablet 88b2d11fdd9547a994a8a85594e767c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:19 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:41400
I20260926 22:48:19.684530 1184 raft_consensus.cc:2229] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:19.684636 1184 raft_consensus.cc:2258] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:19.684767 1185 tablet_replica.cc:330] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: stopping tablet replica
I20260926 22:48:19.684841 1185 raft_consensus.cc:2229] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:19.684926 1185 raft_consensus.cc:2258] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:19.685410 1184 ts_tablet_manager.cc:1905] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:19.685410 1185 ts_tablet_manager.cc:1905] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:19.687572 1185 ts_tablet_manager.cc:1918] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:19.687649 1185 log.cc:1196] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/88b2d11fdd9547a994a8a85594e767c4
I20260926 22:48:19.687822 1184 ts_tablet_manager.cc:1918] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:19.687878 1185 ts_tablet_manager.cc:1939] T 88b2d11fdd9547a994a8a85594e767c4 P f14fb78c84454f318504b3d5c8748699: Deleting consensus metadata
I20260926 22:48:19.687906 1184 log.cc:1196] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/88b2d11fdd9547a994a8a85594e767c4
I20260926 22:48:19.688205 1184 ts_tablet_manager.cc:1939] T 88b2d11fdd9547a994a8a85594e767c4 P ad72461eebf2467b9dd9a7fb03e1ee1c: Deleting consensus metadata
I20260926 22:48:19.688295 671 catalog_manager.cc:4841] TS f14fb78c84454f318504b3d5c8748699 (127.14.184.195:39513): tablet 88b2d11fdd9547a994a8a85594e767c4 (table test-table [id=ddaa0c9b40dc4bf68960d1b51d639a1d]) successfully deleted
I20260926 22:48:19.688581 670 catalog_manager.cc:4841] TS ad72461eebf2467b9dd9a7fb03e1ee1c (127.14.184.194:41597): tablet 88b2d11fdd9547a994a8a85594e767c4 (table test-table [id=ddaa0c9b40dc4bf68960d1b51d639a1d]) successfully deleted
I20260926 22:48:19.691748 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 880
W20260926 22:48:19.691749 668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:42251 (error 108)
I20260926 22:48:19.692638 1156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.692515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:42251 (local address 127.0.0.1:41420)
0926 22:48:19.692564 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.692599 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:42251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 22:48:19.692768 668 catalog_manager.cc:4584] TS 2a750f5e6f9649e6b5af9a3acf94583d (127.14.184.193:42251): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88b2d11fdd9547a994a8a85594e767c4: Network error: Client connection negotiation failed: client connection to 127.14.184.193:42251: connect: Connection refused (error 111)
I20260926 22:48:19.696893 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1014
I20260926 22:48:19.701642 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 652
2026-09-26T22:48:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2289 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T22:48:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:19Z Disabled control of system clock
I20260926 22:48:19.728303 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:42941
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:42941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:19.807715 1190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:19.807924 1190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:19.807961 1190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:19.808996 1190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:19.809027 1190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:19.809039 1190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:19.809052 1190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:19.809064 1190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:19.810299 1190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:42941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:42941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:19.810585 1190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:19.810989 1190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:19.816382 1198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.816434 1195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:19.816447 1190 server_base.cc:1034] running on GCE node
W20260926 22:48:19.816407 1196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:19.816917 1190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:19.817171 1190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:19.818310 1190 hybrid_clock.cc:648] HybridClock initialized: now 1790462899818301 us; error 32 us; skew 500 ppm
I20260926 22:48:19.819569 1190 webserver.cc:460] Webserver started at http://127.14.184.254:45551/ using document root <none> and password file <none>
I20260926 22:48:19.819825 1190 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:19.819872 1190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:19.820021 1190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:19.821058 1190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "3a4c99f9770142b9a3921e99d4b52130"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "5e1f0333d920b9e1a05e220ccb2bea23"
server_key_iv: "ab0bd5f6a28e0e4ab3a8685a064c13f0"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.821422 1190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "3a4c99f9770142b9a3921e99d4b52130"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "5e1f0333d920b9e1a05e220ccb2bea23"
server_key_iv: "ab0bd5f6a28e0e4ab3a8685a064c13f0"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.822777 1190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:19.823541 1204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.823721 1190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:19.823822 1190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "3a4c99f9770142b9a3921e99d4b52130"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "5e1f0333d920b9e1a05e220ccb2bea23"
server_key_iv: "ab0bd5f6a28e0e4ab3a8685a064c13f0"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.823902 1190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:19.841120 1190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:19.841427 1190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:19.841580 1190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:19.845466 1190 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:42941
I20260926 22:48:19.845500 1256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:42941 every 8 connection(s)
I20260926 22:48:19.845898 1190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:19.846915 1257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:19.849113 1257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130: Bootstrap starting.
I20260926 22:48:19.849677 1257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:19.849960 1257 log.cc:824] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:19.850636 1257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130: No bootstrap required, opened a new log
I20260926 22:48:19.852342 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1190
I20260926 22:48:19.852294 1257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a4c99f9770142b9a3921e99d4b52130" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42941 } }
I20260926 22:48:19.852412 1257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:19.852433 1257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a4c99f9770142b9a3921e99d4b52130, State: Initialized, Role: FOLLOWER
I20260926 22:48:19.852464 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:19.852602 1257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a4c99f9770142b9a3921e99d4b52130" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42941 } }
I20260926 22:48:19.852674 1257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:19.852698 15075 external_mini_cluster.cc:1428] Setting key 74352919f30a93cb8a740826e101c009
I20260926 22:48:19.852720 1257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:19.852762 1257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:19.853394 1257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a4c99f9770142b9a3921e99d4b52130" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42941 } }
I20260926 22:48:19.853542 1257 leader_election.cc:302] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a4c99f9770142b9a3921e99d4b52130; no voters:
I20260926 22:48:19.853721 1257 leader_election.cc:288] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:19.853753 1262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:19.853861 1262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [term 1 LEADER]: Becoming Leader. State: Replica: 3a4c99f9770142b9a3921e99d4b52130, State: Running, Role: LEADER
I20260926 22:48:19.853943 1262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a4c99f9770142b9a3921e99d4b52130" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42941 } }
I20260926 22:48:19.853993 1257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:19.854436 1263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a4c99f9770142b9a3921e99d4b52130. Latest consensus state: current_term: 1 leader_uuid: "3a4c99f9770142b9a3921e99d4b52130" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a4c99f9770142b9a3921e99d4b52130" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42941 } } }
I20260926 22:48:19.854573 1263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:19.854449 1264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a4c99f9770142b9a3921e99d4b52130" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a4c99f9770142b9a3921e99d4b52130" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 42941 } } }
I20260926 22:48:19.854753 1264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:19.854818 1271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:19.855319 1271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:19.856948 1271 catalog_manager.cc:1347] Generated new cluster ID: 75b505a391c94f6cbdd5d9bc851299fc
I20260926 22:48:19.857004 1271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:19.859278 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.853225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.14.184.254:42941)
0926 22:48:19.853459 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.853466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.853490 (+ 24us) server_negotiation.cc:407] Connection header received
0926 22:48:19.853618 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.853623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.853703 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.853891 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:19.854084 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.855945 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.856954 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.857519 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.857626 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.858491 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:19.858494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:19.858495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:19.858713 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:19.858750 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:19.858770 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:19.858820 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.859005 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.859077 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":109,"threads_started":1}
I20260926 22:48:19.868062 1271 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:19.868711 1271 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:19.872845 1271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130: Generated new TSK 0
I20260926 22:48:19.873008 1271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:19.875761 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42941
--builtin_ntp_servers=127.14.184.212:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:19.956037 1281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:19.956272 1281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:19.956310 1281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:19.957466 1281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:19.957528 1281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:19.957572 1281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:48:19.959097 1281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:19.959348 1281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:19.959623 1281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:19.965579 1289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.965613 1286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:19.965590 1287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:19.965798 1281 server_base.cc:1034] running on GCE node
I20260926 22:48:19.966002 1281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:19.966252 1281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:19.967411 1281 hybrid_clock.cc:648] HybridClock initialized: now 1790462899967398 us; error 29 us; skew 500 ppm
I20260926 22:48:19.968639 1281 webserver.cc:460] Webserver started at http://127.14.184.193:38283/ using document root <none> and password file <none>
I20260926 22:48:19.968875 1281 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:19.968943 1281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:19.969091 1281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:19.970168 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "4b8f1171155b496db5290d0192408c78"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "6356198104b14f81408a77903f804ba0"
server_key_iv: "508922352e1afcaa75a30d207cf36856"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.970549 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "4b8f1171155b496db5290d0192408c78"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "6356198104b14f81408a77903f804ba0"
server_key_iv: "508922352e1afcaa75a30d207cf36856"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.971918 1281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:19.972715 1295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.972950 1281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:19.973067 1281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "4b8f1171155b496db5290d0192408c78"
format_stamp: "Formatted at 2026-09-26 22:48:19 on dist-test-slave-3986"
server_key: "6356198104b14f81408a77903f804ba0"
server_key_iv: "508922352e1afcaa75a30d207cf36856"
server_key_version: "encryptionkey@0"
I20260926 22:48:19.973161 1281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:19.987419 1281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:19.987675 1281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:19.987783 1281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:19.987962 1281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:19.988229 1281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:19.988257 1281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.988277 1281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:19.988291 1281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:19.993721 1281 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:33551
I20260926 22:48:19.993822 1408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:33551 every 8 connection(s)
I20260926 22:48:19.994115 1281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:48:19.999294 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.994906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:60529 (local address 127.14.184.254:42941)
0926 22:48:19.994989 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:19.994991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:19.995199 (+ 208us) server_negotiation.cc:407] Connection header received
0926 22:48:19.995708 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:19.995709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:19.995722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:19.995782 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:19.996475 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.996966 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.997941 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.998135 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.998207 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.998994 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:19.998996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:19.998996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:19.999000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:19.999015 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:19.999019 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:19.999073 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:19.999178 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 22:48:19.999208 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 22:48:19.999486 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:19.994886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:42941 (local address 127.14.184.193:60529)
0926 22:48:19.995081 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:19.995113 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:19.995577 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:19.995891 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:19.995895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:19.995903 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:19.996378 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.996382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.997063 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.997064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.997846 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:19.997849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:19.998230 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:19.998231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:19.998365 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:19.998858 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:19.998858 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:19.998884 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:19.998905 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:19.999078 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:19.999082 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:19.999085 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:19.999165 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 22:48:19.999264 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260926 22:48:20.000028 1409 heartbeater.cc:347] Connected to a master server at 127.14.184.254:42941
I20260926 22:48:20.000114 1409 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:20.000303 1409 heartbeater.cc:511] Master 127.14.184.254:42941 requested a full tablet report, sending...
I20260926 22:48:20.000669 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1281
I20260926 22:48:20.000775 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:48:20.000869 1221 ts_manager.cc:194] Registered new tserver with Master: 4b8f1171155b496db5290d0192408c78 (127.14.184.193:33551)
I20260926 22:48:20.000967 15075 external_mini_cluster.cc:1428] Setting key 497c33ab2e9b65ab6aa05dba15aa618a
I20260926 22:48:20.001514 1221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.193:60529
I20260926 22:48:20.001972 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42941
--builtin_ntp_servers=127.14.184.212:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:20.080977 1412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:20.081171 1412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:20.081192 1412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:20.082232 1412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:20.082263 1412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:20.082288 1412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:48:20.083535 1412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:20.083712 1412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:20.083942 1412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:20.088670 1418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:20.088665 1420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:20.088665 1417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:20.088761 1412 server_base.cc:1034] running on GCE node
I20260926 22:48:20.089155 1412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:20.089403 1412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:20.090570 1412 hybrid_clock.cc:648] HybridClock initialized: now 1790462900090561 us; error 38 us; skew 500 ppm
I20260926 22:48:20.091872 1412 webserver.cc:460] Webserver started at http://127.14.184.194:45731/ using document root <none> and password file <none>
I20260926 22:48:20.092121 1412 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:20.092211 1412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:20.092360 1412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:20.093376 1412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "29313d2cb88f406585fadbbe6c170ae3"
format_stamp: "Formatted at 2026-09-26 22:48:20 on dist-test-slave-3986"
server_key: "bd9ab9a2998ed26fe7a0e92ff086ad31"
server_key_iv: "f375dadcec50d78cdb53e1b4d6ca82ea"
server_key_version: "encryptionkey@0"
I20260926 22:48:20.093750 1412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "29313d2cb88f406585fadbbe6c170ae3"
format_stamp: "Formatted at 2026-09-26 22:48:20 on dist-test-slave-3986"
server_key: "bd9ab9a2998ed26fe7a0e92ff086ad31"
server_key_iv: "f375dadcec50d78cdb53e1b4d6ca82ea"
server_key_version: "encryptionkey@0"
I20260926 22:48:20.095157 1412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:20.095837 1426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.096055 1412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.096146 1412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "29313d2cb88f406585fadbbe6c170ae3"
format_stamp: "Formatted at 2026-09-26 22:48:20 on dist-test-slave-3986"
server_key: "bd9ab9a2998ed26fe7a0e92ff086ad31"
server_key_iv: "f375dadcec50d78cdb53e1b4d6ca82ea"
server_key_version: "encryptionkey@0"
I20260926 22:48:20.096231 1412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:20.107483 1412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:20.107718 1412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:20.107821 1412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:20.107978 1412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:20.108229 1412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:20.108258 1412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.108279 1412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:20.108291 1412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.113822 1412 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:43053
I20260926 22:48:20.113862 1539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:43053 every 8 connection(s)
I20260926 22:48:20.114169 1412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:48:20.116570 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1412
I20260926 22:48:20.116683 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:48:20.116873 15075 external_mini_cluster.cc:1428] Setting key 97b09388b3a4f845cd8ac305daac871b
I20260926 22:48:20.117784 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42941
--builtin_ntp_servers=127.14.184.212:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 22:48:20.119748 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.114922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:45197 (local address 127.14.184.254:42941)
0926 22:48:20.114965 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.114968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.115209 (+ 241us) server_negotiation.cc:407] Connection header received
0926 22:48:20.115673 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.115674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.115683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.115761 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:20.116402 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.117167 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.118274 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.118487 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.118573 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.119475 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:20.119477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:20.119477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:20.119481 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:20.119495 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:20.119499 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:20.119545 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.119648 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.119670 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:20.120074 1542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.114887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:42941 (local address 127.14.184.194:45197)
0926 22:48:20.115062 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.115102 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.115575 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.115814 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.115817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.115825 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:20.116309 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.116313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.117273 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.117274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.118163 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.118166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.118585 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.118586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.118769 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.119317 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:20.119318 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:20.119357 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:20.119385 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:20.119568 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:20.119572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:20.119575 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.119646 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.119752 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260926 22:48:20.120527 1540 heartbeater.cc:347] Connected to a master server at 127.14.184.254:42941
I20260926 22:48:20.120632 1540 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:20.120812 1540 heartbeater.cc:511] Master 127.14.184.254:42941 requested a full tablet report, sending...
I20260926 22:48:20.121183 1221 ts_manager.cc:194] Registered new tserver with Master: 29313d2cb88f406585fadbbe6c170ae3 (127.14.184.194:43053)
I20260926 22:48:20.121716 1221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.194:45197
W20260926 22:48:20.198345 1543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:20.198614 1543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:20.198652 1543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:20.199918 1543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:20.199971 1543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:20.200013 1543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:48:20.201336 1543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:42941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:20.201589 1543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:20.201892 1543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:20.206668 1548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:20.206729 1551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:20.206785 1543 server_base.cc:1034] running on GCE node
W20260926 22:48:20.206647 1549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:20.207098 1543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:20.207294 1543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:20.208433 1543 hybrid_clock.cc:648] HybridClock initialized: now 1790462900208424 us; error 31 us; skew 500 ppm
I20260926 22:48:20.209594 1543 webserver.cc:460] Webserver started at http://127.14.184.195:42357/ using document root <none> and password file <none>
I20260926 22:48:20.209864 1543 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:20.209935 1543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:20.210085 1543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:20.211200 1543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "b4e5e26d0fea462eadb68bea2805e301"
format_stamp: "Formatted at 2026-09-26 22:48:20 on dist-test-slave-3986"
server_key: "fd7f136cee3aee10b745fce0c414c79a"
server_key_iv: "803a40d2209fcca7d1a06b975e7e4536"
server_key_version: "encryptionkey@0"
I20260926 22:48:20.211544 1543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "b4e5e26d0fea462eadb68bea2805e301"
format_stamp: "Formatted at 2026-09-26 22:48:20 on dist-test-slave-3986"
server_key: "fd7f136cee3aee10b745fce0c414c79a"
server_key_iv: "803a40d2209fcca7d1a06b975e7e4536"
server_key_version: "encryptionkey@0"
I20260926 22:48:20.212886 1543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:20.213551 1557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.213713 1543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:20.213814 1543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "b4e5e26d0fea462eadb68bea2805e301"
format_stamp: "Formatted at 2026-09-26 22:48:20 on dist-test-slave-3986"
server_key: "fd7f136cee3aee10b745fce0c414c79a"
server_key_iv: "803a40d2209fcca7d1a06b975e7e4536"
server_key_version: "encryptionkey@0"
I20260926 22:48:20.213889 1543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:20.231177 1543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:20.231464 1543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:20.231611 1543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:20.231828 1543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:20.232144 1543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:20.232205 1543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.232241 1543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:20.232268 1543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:20.237950 1543 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:44555
I20260926 22:48:20.237998 1670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:44555 every 8 connection(s)
I20260926 22:48:20.238307 1543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:48:20.242910 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1543
I20260926 22:48:20.243034 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:48:20.243120 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.238994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:38281 (local address 127.14.184.254:42941)
0926 22:48:20.239052 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.239054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.239308 (+ 254us) server_negotiation.cc:407] Connection header received
0926 22:48:20.239772 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.239773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.239782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.239834 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:20.240443 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.240923 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.241839 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.242019 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.242068 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.242817 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:20.242819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:20.242819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:20.242824 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:20.242843 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:20.242848 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:20.242912 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.242991 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.243029 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 22:48:20.243273 15075 external_mini_cluster.cc:1428] Setting key d7553946c410c43a9d6fd6caee3eedb0
I20260926 22:48:20.243358 1673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.238985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:42941 (local address 127.14.184.195:38281)
0926 22:48:20.239173 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.239218 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.239682 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.239884 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.239887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.239895 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:20.240371 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.240375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.240999 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.241000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.241761 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.241763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.242089 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.242089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.242202 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.242647 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:20.242647 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:20.242690 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:20.242711 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:20.242917 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:20.242921 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:20.242923 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.242972 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.243090 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260926 22:48:20.243974 1671 heartbeater.cc:347] Connected to a master server at 127.14.184.254:42941
I20260926 22:48:20.244063 1671 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:20.244232 1671 heartbeater.cc:511] Master 127.14.184.254:42941 requested a full tablet report, sending...
I20260926 22:48:20.244550 1221 ts_manager.cc:194] Registered new tserver with Master: b4e5e26d0fea462eadb68bea2805e301 (127.14.184.195:44555)
I20260926 22:48:20.244910 1221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.195:38281
I20260926 22:48:20.255180 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:48:20.259333 1261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.256559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.14.184.254:42941)
0926 22:48:20.256607 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.256609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.256728 (+ 119us) server_negotiation.cc:407] Connection header received
0926 22:48:20.256787 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.256788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.256798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.256845 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:20.257009 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.257477 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.258169 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.258348 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.258397 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.259027 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:20.259028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:20.259028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:20.259032 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:20.259045 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:20.259049 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:20.259122 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.259183 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.259193 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:48:20.260855 1221 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:48:20.262339 1221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:48:20.268390 1680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.264433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:43053 (local address 127.0.0.1:55210)
0926 22:48:20.264746 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.264784 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.264885 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.265093 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.265097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.265099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:20.265168 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.265169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.266163 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.266164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.267228 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.267230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.268106 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.268107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.268175 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.268188 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.268270 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.268320 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":61,"threads_started":1}
I20260926 22:48:20.268402 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.264514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.14.184.194:43053)
0926 22:48:20.264712 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.264718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.264882 (+ 164us) server_negotiation.cc:407] Connection header received
0926 22:48:20.264979 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.264981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.264993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.265063 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:20.265271 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.266078 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.267322 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.268009 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.268105 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.268195 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.268283 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.268320 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260926 22:48:20.268903 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.264448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:44555 (local address 127.0.0.1:58176)
0926 22:48:20.265139 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.265148 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.265188 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.265437 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.265438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.265440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:20.265476 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.265478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.266787 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.266788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.267645 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.267647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.268653 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.268654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.268730 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.268734 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.268782 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.268823 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260926 22:48:20.269001 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.264433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:33551 (local address 127.0.0.1:56350)
0926 22:48:20.265003 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.265011 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.265057 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.265353 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.265354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.265356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:20.265401 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.265403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.266699 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.266700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.267743 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.267746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.268733 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.268734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.268795 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.268800 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.268866 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.268908 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260926 22:48:20.269160 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.264699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.14.184.195:44555)
0926 22:48:20.264973 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.264977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.265247 (+ 270us) server_negotiation.cc:407] Connection header received
0926 22:48:20.265288 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.265290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.265307 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.265375 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:20.265539 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.266629 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.267730 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.268450 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.268531 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.268614 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.269013 (+ 399us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.269060 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":96,"threads_started":1}
I20260926 22:48:20.269240 1474 tablet_service.cc:1467] Processing CreateTablet for tablet f8922d1def53403ba32b4daa6469a810 (DEFAULT_TABLE table=test-table [id=072aa2b345a742d3999cce1224f2cb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:20.269387 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.264711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.14.184.193:33551)
0926 22:48:20.264886 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.264890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.265052 (+ 162us) server_negotiation.cc:407] Connection header received
0926 22:48:20.265170 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.265173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.265188 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.265274 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:20.265479 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.266584 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.267823 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.268650 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.268719 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.268799 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.268879 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.268916 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260926 22:48:20.269615 1474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8922d1def53403ba32b4daa6469a810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:20.269742 1605 tablet_service.cc:1467] Processing CreateTablet for tablet f8922d1def53403ba32b4daa6469a810 (DEFAULT_TABLE table=test-table [id=072aa2b345a742d3999cce1224f2cb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:20.270051 1605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8922d1def53403ba32b4daa6469a810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:20.273141 1685 tablet_bootstrap.cc:492] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Bootstrap starting.
I20260926 22:48:20.273528 1343 tablet_service.cc:1467] Processing CreateTablet for tablet f8922d1def53403ba32b4daa6469a810 (DEFAULT_TABLE table=test-table [id=072aa2b345a742d3999cce1224f2cb0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:20.273787 1343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8922d1def53403ba32b4daa6469a810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:20.273830 1685 tablet_bootstrap.cc:654] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:20.274102 1685 log.cc:824] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:20.274910 1685 tablet_bootstrap.cc:492] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: No bootstrap required, opened a new log
I20260926 22:48:20.274989 1685 ts_tablet_manager.cc:1397] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:20.276489 1688 tablet_bootstrap.cc:492] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78: Bootstrap starting.
I20260926 22:48:20.276945 1686 tablet_bootstrap.cc:492] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Bootstrap starting.
I20260926 22:48:20.277211 1688 tablet_bootstrap.cc:654] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:20.277521 1688 log.cc:824] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:20.277503 1685 raft_consensus.cc:348] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.277630 1686 tablet_bootstrap.cc:654] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:20.277666 1685 raft_consensus.cc:374] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:20.277730 1685 raft_consensus.cc:729] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29313d2cb88f406585fadbbe6c170ae3, State: Initialized, Role: FOLLOWER
I20260926 22:48:20.277871 1685 consensus_queue.cc:260] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.277935 1686 log.cc:824] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:20.278146 1685 ts_tablet_manager.cc:1428] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:48:20.278240 1688 tablet_bootstrap.cc:492] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78: No bootstrap required, opened a new log
I20260926 22:48:20.278282 1540 heartbeater.cc:503] Master 127.14.184.254:42941 was elected leader, sending a full tablet report...
I20260926 22:48:20.278307 1688 ts_tablet_manager.cc:1397] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:20.278805 1686 tablet_bootstrap.cc:492] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: No bootstrap required, opened a new log
I20260926 22:48:20.278882 1686 ts_tablet_manager.cc:1397] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:20.280273 1688 raft_consensus.cc:348] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.280453 1688 raft_consensus.cc:374] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:20.280496 1688 raft_consensus.cc:729] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b8f1171155b496db5290d0192408c78, State: Initialized, Role: FOLLOWER
I20260926 22:48:20.280624 1688 consensus_queue.cc:260] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.280875 1688 ts_tablet_manager.cc:1428] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:48:20.280925 1409 heartbeater.cc:503] Master 127.14.184.254:42941 was elected leader, sending a full tablet report...
I20260926 22:48:20.281160 1686 raft_consensus.cc:348] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.281312 1686 raft_consensus.cc:374] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:20.281356 1686 raft_consensus.cc:729] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4e5e26d0fea462eadb68bea2805e301, State: Initialized, Role: FOLLOWER
I20260926 22:48:20.281491 1686 consensus_queue.cc:260] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.281769 1686 ts_tablet_manager.cc:1428] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:48:20.281782 1671 heartbeater.cc:503] Master 127.14.184.254:42941 was elected leader, sending a full tablet report...
I20260926 22:48:20.312336 1690 raft_consensus.cc:482] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:20.312489 1690 raft_consensus.cc:504] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.312863 1690 leader_election.cc:288] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b8f1171155b496db5290d0192408c78 (127.14.184.193:33551), b4e5e26d0fea462eadb68bea2805e301 (127.14.184.195:44555)
I20260926 22:48:20.316536 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.313035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:44555 (local address 127.14.184.194:50129)
0926 22:48:20.313391 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.313402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.313458 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.313600 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.313601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.313602 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:20.313653 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.313654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.314491 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.314491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.315605 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.315607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.316312 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.316313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.316372 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.316375 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.316420 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.316449 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":148,"thread_start_us":137,"threads_started":1}
I20260926 22:48:20.316570 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.313112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:50129 (local address 127.14.184.195:44555)
0926 22:48:20.313187 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.313189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.313497 (+ 308us) server_negotiation.cc:407] Connection header received
0926 22:48:20.313531 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.313532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.313541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.313590 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:20.313729 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.314422 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.315690 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.316238 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.316307 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.316314 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.316452 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.316497 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 22:48:20.316690 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.313131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:41607 (local address 127.14.184.193:33551)
0926 22:48:20.313177 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.313178 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.313187 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:20.313215 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.313216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.313224 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.313280 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:20.313486 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.314250 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.315656 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.316232 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.316307 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.316314 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.316560 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.316607 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 22:48:20.316642 1542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.313035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:33551 (local address 127.14.184.194:41607)
0926 22:48:20.313077 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.313089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:20.313130 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:20.313296 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:20.313297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:20.313299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:20.313387 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.313388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.314325 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.314326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.315544 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:20.315547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.316372 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.316373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:20.316461 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.316465 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:20.316529 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 22:48:20.316570 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 22:48:20.316973 1625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8922d1def53403ba32b4daa6469a810" candidate_uuid: "29313d2cb88f406585fadbbe6c170ae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4e5e26d0fea462eadb68bea2805e301" is_pre_election: true
I20260926 22:48:20.317049 1363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8922d1def53403ba32b4daa6469a810" candidate_uuid: "29313d2cb88f406585fadbbe6c170ae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8f1171155b496db5290d0192408c78" is_pre_election: true
I20260926 22:48:20.317137 1625 raft_consensus.cc:2454] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29313d2cb88f406585fadbbe6c170ae3 in term 0.
I20260926 22:48:20.317157 1363 raft_consensus.cc:2454] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29313d2cb88f406585fadbbe6c170ae3 in term 0.
I20260926 22:48:20.317365 1428 leader_election.cc:302] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29313d2cb88f406585fadbbe6c170ae3, 4b8f1171155b496db5290d0192408c78; no voters:
I20260926 22:48:20.317512 1690 raft_consensus.cc:2781] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:48:20.317564 1690 raft_consensus.cc:482] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:20.317600 1690 raft_consensus.cc:3037] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:20.318267 1690 raft_consensus.cc:504] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.318406 1690 leader_election.cc:288] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [CANDIDATE]: Term 1 election: Requested vote from peers 4b8f1171155b496db5290d0192408c78 (127.14.184.193:33551), b4e5e26d0fea462eadb68bea2805e301 (127.14.184.195:44555)
I20260926 22:48:20.318549 1363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8922d1def53403ba32b4daa6469a810" candidate_uuid: "29313d2cb88f406585fadbbe6c170ae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8f1171155b496db5290d0192408c78"
I20260926 22:48:20.318557 1625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8922d1def53403ba32b4daa6469a810" candidate_uuid: "29313d2cb88f406585fadbbe6c170ae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4e5e26d0fea462eadb68bea2805e301"
I20260926 22:48:20.318634 1625 raft_consensus.cc:3037] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:20.318634 1363 raft_consensus.cc:3037] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:20.319562 1363 raft_consensus.cc:2454] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29313d2cb88f406585fadbbe6c170ae3 in term 1.
I20260926 22:48:20.319562 1625 raft_consensus.cc:2454] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29313d2cb88f406585fadbbe6c170ae3 in term 1.
I20260926 22:48:20.319770 1430 leader_election.cc:302] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29313d2cb88f406585fadbbe6c170ae3, b4e5e26d0fea462eadb68bea2805e301; no voters:
I20260926 22:48:20.319896 1690 raft_consensus.cc:2781] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:20.320077 1690 raft_consensus.cc:686] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 1 LEADER]: Becoming Leader. State: Replica: 29313d2cb88f406585fadbbe6c170ae3, State: Running, Role: LEADER
I20260926 22:48:20.320184 1690 consensus_queue.cc:237] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } }
I20260926 22:48:20.320961 1221 catalog_manager.cc:5495] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29313d2cb88f406585fadbbe6c170ae3 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "29313d2cb88f406585fadbbe6c170ae3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29313d2cb88f406585fadbbe6c170ae3" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 43053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260926 22:48:20.326184 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.323863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55222 (local address 127.14.184.194:43053)
0926 22:48:20.323921 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:20.323923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:20.323942 (+ 19us) server_negotiation.cc:407] Connection header received
0926 22:48:20.323969 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:20.323969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:20.323985 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:20.324060 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:20.324198 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.324625 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.325348 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:20.325570 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:20.325654 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:20.325831 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:20.325936 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:20.326063 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:20.326094 (+ 31us) server_negotiation.cc:299] Negotiation successful
0926 22:48:20.326101 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:48:20.327991 1363 raft_consensus.cc:1261] T f8922d1def53403ba32b4daa6469a810 P 4b8f1171155b496db5290d0192408c78 [term 1 FOLLOWER]: Refusing update from remote peer 29313d2cb88f406585fadbbe6c170ae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:20.328032 1625 raft_consensus.cc:1261] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 1 FOLLOWER]: Refusing update from remote peer 29313d2cb88f406585fadbbe6c170ae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:20.328315 1690 consensus_queue.cc:1035] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b8f1171155b496db5290d0192408c78" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 33551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:20.328469 1695 consensus_queue.cc:1035] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4e5e26d0fea462eadb68bea2805e301" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:20.330739 1698 mvcc.cc:205] Tried to move back new op lower bound from 7333736039740964864 to 7333736039712227328. Current Snapshot: MvccSnapshot[applied={T|T < 7333736039740964864}]
I20260926 22:48:20.330919 1699 mvcc.cc:205] Tried to move back new op lower bound from 7333736039740964864 to 7333736039712227328. Current Snapshot: MvccSnapshot[applied={T|T < 7333736039740964864}]
I20260926 22:48:20.331048 1700 mvcc.cc:205] Tried to move back new op lower bound from 7333736039740964864 to 7333736039712227328. Current Snapshot: MvccSnapshot[applied={T|T < 7333736039740964864}]
I20260926 22:48:20.333297 1220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:20.333405 1220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:20.334115 1220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a4c99f9770142b9a3921e99d4b52130: Sending DeleteTablet for 3 replicas of tablet f8922d1def53403ba32b4daa6469a810
I20260926 22:48:20.334442 1474 tablet_service.cc:1514] Processing DeleteTablet for tablet f8922d1def53403ba32b4daa6469a810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:20 UTC) from {username='slave'} at 127.0.0.1:55210
I20260926 22:48:20.334457 1605 tablet_service.cc:1514] Processing DeleteTablet for tablet f8922d1def53403ba32b4daa6469a810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:20 UTC) from {username='slave'} at 127.0.0.1:58176
I20260926 22:48:20.334723 1709 tablet_replica.cc:330] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: stopping tablet replica
I20260926 22:48:20.334764 1708 tablet_replica.cc:330] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: stopping tablet replica
I20260926 22:48:20.334740 1343 tablet_service.cc:1514] Processing DeleteTablet for tablet f8922d1def53403ba32b4daa6469a810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:20 UTC) from {username='slave'} at 127.0.0.1:56350
I20260926 22:48:20.334857 1708 raft_consensus.cc:2229] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:20.334951 1709 raft_consensus.cc:2229] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:48:20.334964 1708 raft_consensus.cc:2258] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:20.335201 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1281
I20260926 22:48:20.335211 1709 raft_consensus.cc:2258] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:20.335371 1708 ts_tablet_manager.cc:1905] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:20.336033 1709 ts_tablet_manager.cc:1905] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:20.337044 1708 ts_tablet_manager.cc:1918] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:20.337118 1708 log.cc:1196] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/f8922d1def53403ba32b4daa6469a810
I20260926 22:48:20.337375 1708 ts_tablet_manager.cc:1939] T f8922d1def53403ba32b4daa6469a810 P b4e5e26d0fea462eadb68bea2805e301: Deleting consensus metadata
I20260926 22:48:20.337730 1709 ts_tablet_manager.cc:1918] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:20.337805 1709 log.cc:1196] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/f8922d1def53403ba32b4daa6469a810
I20260926 22:48:20.337783 1208 catalog_manager.cc:4841] TS b4e5e26d0fea462eadb68bea2805e301 (127.14.184.195:44555): tablet f8922d1def53403ba32b4daa6469a810 (table test-table [id=072aa2b345a742d3999cce1224f2cb0b]) successfully deleted
I20260926 22:48:20.338034 1709 ts_tablet_manager.cc:1939] T f8922d1def53403ba32b4daa6469a810 P 29313d2cb88f406585fadbbe6c170ae3: Deleting consensus metadata
I20260926 22:48:20.338475 1205 catalog_manager.cc:4841] TS 29313d2cb88f406585fadbbe6c170ae3 (127.14.184.194:43053): tablet f8922d1def53403ba32b4daa6469a810 (table test-table [id=072aa2b345a742d3999cce1224f2cb0b]) successfully deleted
W20260926 22:48:20.341650 1206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:33551 (error 108)
I20260926 22:48:20.341782 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1412
I20260926 22:48:20.342657 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:20.342524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:33551 (local address 127.0.0.1:56364)
0926 22:48:20.342581 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:20.342620 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 22:48:20.343164 1206 catalog_manager.cc:4584] TS 4b8f1171155b496db5290d0192408c78 (127.14.184.193:33551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8922d1def53403ba32b4daa6469a810: Network error: Client connection negotiation failed: client connection to 127.14.184.193:33551: connect: Connection refused (error 111)
I20260926 22:48:20.346360 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1543
I20260926 22:48:20.350636 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1190
2026-09-26T22:48:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (648 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:20 dist-test-slave-3986 krb5kdc[1714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:20 dist-test-slave-3986 krb5kdc[1714](info): set up 2 sockets
Sep 26 22:48:20 dist-test-slave-3986 krb5kdc[1714](info): commencing operation
krb5kdc: starting...
W20260926 22:48:21.374832 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:21.486523 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34585
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:34585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:21.564793 1727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:21.564992 1727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:21.565013 1727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:21.566036 1727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:21.566071 1727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:21.566085 1727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:21.566097 1727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:21.566109 1727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:21.567492 1727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:34585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:21.567662 1727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:21.567816 1727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:21.572558 1733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:21.572669 1727 server_base.cc:1034] running on GCE node
W20260926 22:48:21.572598 1732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:21.572738 1735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:21.573221 1727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:21.573490 1727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:21.574724 1727 hybrid_clock.cc:648] HybridClock initialized: now 1790462901574712 us; error 38 us; skew 500 ppm
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:21.580502 1727 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:21.580917 1727 webserver.cc:460] Webserver started at http://127.14.184.254:44437/ using document root <none> and password file <none>
I20260926 22:48:21.581136 1727 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:21.581179 1727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:21.581354 1727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:21.582404 1727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "1e19e65da2c8428098868f6160bfc0b0"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "49f9981e8af6be58228339b8ad6919f4"
server_key_iv: "77e0248d19855570c98544c7910d831a"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.582770 1727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "1e19e65da2c8428098868f6160bfc0b0"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "49f9981e8af6be58228339b8ad6919f4"
server_key_iv: "77e0248d19855570c98544c7910d831a"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.584048 1727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:21.584777 1742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.584952 1727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:21.585040 1727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "1e19e65da2c8428098868f6160bfc0b0"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "49f9981e8af6be58228339b8ad6919f4"
server_key_iv: "77e0248d19855570c98544c7910d831a"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.585111 1727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:21.594890 1727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:21.596586 1727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:21.596717 1727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:21.600531 1727 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:34585
I20260926 22:48:21.600594 1794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:34585 every 8 connection(s)
I20260926 22:48:21.600922 1727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:21.601737 1795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:21.603999 1795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0: Bootstrap starting.
I20260926 22:48:21.604555 1795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:21.604837 1795 log.cc:824] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:21.605477 1795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0: No bootstrap required, opened a new log
I20260926 22:48:21.607081 1795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19e65da2c8428098868f6160bfc0b0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34585 } }
I20260926 22:48:21.607244 1795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:21.607291 1795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e19e65da2c8428098868f6160bfc0b0, State: Initialized, Role: FOLLOWER
I20260926 22:48:21.607450 1795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19e65da2c8428098868f6160bfc0b0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34585 } }
I20260926 22:48:21.607546 1795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:21.607589 1795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:21.607638 1795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:21.608121 1795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19e65da2c8428098868f6160bfc0b0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34585 } }
I20260926 22:48:21.608243 1795 leader_election.cc:302] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e19e65da2c8428098868f6160bfc0b0; no voters:
I20260926 22:48:21.608414 1795 leader_election.cc:288] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:21.608451 1798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:21.608608 1798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [term 1 LEADER]: Becoming Leader. State: Replica: 1e19e65da2c8428098868f6160bfc0b0, State: Running, Role: LEADER
I20260926 22:48:21.608665 1795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:21.608685 1798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19e65da2c8428098868f6160bfc0b0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34585 } }
I20260926 22:48:21.609072 1800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e19e65da2c8428098868f6160bfc0b0. Latest consensus state: current_term: 1 leader_uuid: "1e19e65da2c8428098868f6160bfc0b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19e65da2c8428098868f6160bfc0b0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34585 } } }
I20260926 22:48:21.609151 1800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:21.609208 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e19e65da2c8428098868f6160bfc0b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19e65da2c8428098868f6160bfc0b0" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34585 } } }
I20260926 22:48:21.609306 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:21.609858 1805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:21.610330 1805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:21.610607 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1727
I20260926 22:48:21.610744 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:21.610931 15075 external_mini_cluster.cc:1428] Setting key 63d3b234a0dc947208a91392874333de
I20260926 22:48:21.611827 1805 catalog_manager.cc:1347] Generated new cluster ID: 9afb681f7d394480ac501af6f8c5d45e
I20260926 22:48:21.611879 1805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:21.625068 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:21.611367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.14.184.254:34585)
0926 22:48:21.611692 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:21.611700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:21.611729 (+ 29us) server_negotiation.cc:407] Connection header received
0926 22:48:21.614792 (+ 3063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:21.614800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:21.614876 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:21.615025 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:21.615745 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.617106 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.618187 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.618785 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.618880 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:21.621524 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:21.621528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:21.621529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:21.621540 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:21.623472 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:21.623847 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:21.623850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:21.623850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:21.623954 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:21.624157 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:21.624157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:21.624158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:21.624466 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:21.624591 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:21.624700 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 22:48:21.624837 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":284,"thread_start_us":71,"threads_started":1}
I20260926 22:48:21.628470 1805 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:21.628929 1805 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:21.646579 1805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0: Generated new TSK 0
I20260926 22:48:21.646807 1805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.184.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
Entry for principal kudu/127.14.184.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab.
I20260926 22:48:21.675262 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34585
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:21.754732 1821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:21.754951 1821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:21.754988 1821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:21.756150 1821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:21.756215 1821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:21.756255 1821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:48:21.757757 1821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.193.keytab
--principal=kudu/127.14.184.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:21.758005 1821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:21.758301 1821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:21.763098 1827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:21.763068 1829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:21.763063 1826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:21.763211 1821 server_base.cc:1034] running on GCE node
I20260926 22:48:21.763454 1821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:21.763757 1821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:21.764958 1821 hybrid_clock.cc:648] HybridClock initialized: now 1790462901764936 us; error 33 us; skew 500 ppm
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:21.767351 1821 init.cc:377] Logged in from keytab as kudu/127.14.184.193@KRBTEST.COM (short username kudu)
I20260926 22:48:21.767752 1821 webserver.cc:460] Webserver started at http://127.14.184.193:42333/ using document root <none> and password file <none>
I20260926 22:48:21.767982 1821 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:21.768049 1821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:21.768190 1821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:21.769169 1821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "acdd65e5b44b4323b0ac880883e75754"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "d8ec2f5fc172ca1cf121668ee0af2387"
server_key_iv: "7a2f83fdc20c583422553309781aab1f"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.769538 1821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "acdd65e5b44b4323b0ac880883e75754"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "d8ec2f5fc172ca1cf121668ee0af2387"
server_key_iv: "7a2f83fdc20c583422553309781aab1f"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.770905 1821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:21.771674 1836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.771865 1821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:21.771952 1821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "acdd65e5b44b4323b0ac880883e75754"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "d8ec2f5fc172ca1cf121668ee0af2387"
server_key_iv: "7a2f83fdc20c583422553309781aab1f"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.772028 1821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:21.783668 1821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:21.785511 1821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:21.785676 1821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:21.785939 1821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:21.786237 1821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:21.786288 1821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.786319 1821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:21.786346 1821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.792305 1821 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:38429
I20260926 22:48:21.792354 1949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:38429 every 8 connection(s)
I20260926 22:48:21.792654 1821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.193@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:21.800221 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1821
I20260926 22:48:21.800348 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:48:21.800534 15075 external_mini_cluster.cc:1428] Setting key f2c60575eb58e036db0b4ca4ca8509ad
I20260926 22:48:21.801811 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:21.793335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:32983 (local address 127.14.184.254:34585)
0926 22:48:21.793399 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:21.793401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:21.793591 (+ 190us) server_negotiation.cc:407] Connection header received
0926 22:48:21.794087 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:21.794088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:21.794101 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:21.794149 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:21.795115 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.795606 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.796520 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.796717 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.796769 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:21.798803 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:21.798804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:21.798805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:21.798811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:21.800250 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:21.800698 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:21.800699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:21.800699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:21.800758 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:21.801071 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:21.801071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:21.801072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:21.801249 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:21.801319 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:21.801596 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 22:48:21.801719 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 22:48:21.801939 1952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:21.793317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34585 (local address 127.14.184.193:32983)
0926 22:48:21.793495 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:21.793528 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:21.793996 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:21.794204 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:21.794207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:21.794855 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:21.795040 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:21.795045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.795682 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.795683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:21.796448 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:21.796450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.796794 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.796794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:21.796889 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:21.797338 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:21.797345 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:21.798720 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:21.800356 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:21.800364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:21.800366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:21.800633 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:21.800841 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:21.800842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:21.800842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:21.801007 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:21.801324 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:21.801332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:21.801464 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 22:48:21.801572 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 22:48:21.801721 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260926 22:48:21.802368 1950 heartbeater.cc:347] Connected to a master server at 127.14.184.254:34585
I20260926 22:48:21.802465 1950 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:21.802683 1950 heartbeater.cc:511] Master 127.14.184.254:34585 requested a full tablet report, sending...
I20260926 22:48:21.803225 1759 ts_manager.cc:194] Registered new tserver with Master: acdd65e5b44b4323b0ac880883e75754 (127.14.184.193:38429)
I20260926 22:48:21.803993 1759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.193@KRBTEST.COM'} at 127.14.184.193:32983
WARNING: no policy specified for kudu/127.14.184.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
Entry for principal kudu/127.14.184.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab.
I20260926 22:48:21.823706 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34585
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:21.903427 1955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:21.903620 1955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:21.903644 1955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:21.904702 1955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:21.904733 1955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:21.904757 1955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:48:21.906065 1955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.194.keytab
--principal=kudu/127.14.184.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:21.906251 1955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:21.906405 1955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:21.911465 1960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:21.911496 1963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:21.911466 1961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:21.911684 1955 server_base.cc:1034] running on GCE node
I20260926 22:48:21.911893 1955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:21.912178 1955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:21.913334 1955 hybrid_clock.cc:648] HybridClock initialized: now 1790462901913318 us; error 29 us; skew 500 ppm
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:21.915710 1955 init.cc:377] Logged in from keytab as kudu/127.14.184.194@KRBTEST.COM (short username kudu)
I20260926 22:48:21.916145 1955 webserver.cc:460] Webserver started at http://127.14.184.194:33765/ using document root <none> and password file <none>
I20260926 22:48:21.916371 1955 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:21.916440 1955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:21.916584 1955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:21.917568 1955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "5cb222f435b94bb4bfd19bbfde347ec0"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "9d541e2d56792e606fa7149bab824888"
server_key_iv: "400f1f51b3e9c55be1442af9512df736"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.917938 1955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "5cb222f435b94bb4bfd19bbfde347ec0"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "9d541e2d56792e606fa7149bab824888"
server_key_iv: "400f1f51b3e9c55be1442af9512df736"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.919269 1955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:21.919953 1970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.920116 1955 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:21.920222 1955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "5cb222f435b94bb4bfd19bbfde347ec0"
format_stamp: "Formatted at 2026-09-26 22:48:21 on dist-test-slave-3986"
server_key: "9d541e2d56792e606fa7149bab824888"
server_key_iv: "400f1f51b3e9c55be1442af9512df736"
server_key_version: "encryptionkey@0"
I20260926 22:48:21.920295 1955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:21.936132 1955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:21.937848 1955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:21.938024 1955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:21.938259 1955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:21.938563 1955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:21.938616 1955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.938654 1955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:21.938699 1955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:21.944428 1955 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:37255
I20260926 22:48:21.944479 2083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:37255 every 8 connection(s)
I20260926 22:48:21.944773 1955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:48:21.948239 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 1955
I20260926 22:48:21.948374 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:48:21.948551 15075 external_mini_cluster.cc:1428] Setting key b77e34077c53044a458d3eb181a862a2
Sep 26 22:48:21 dist-test-slave-3986 krb5kdc[1714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462901, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.194@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:21.954517 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:21.945429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34585 (local address 127.14.184.194:44563)
0926 22:48:21.945578 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:21.945618 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:21.946077 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:21.946276 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:21.946279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:21.946875 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:21.947065 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:21.947070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.947726 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.947726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:21.948633 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:21.948636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.949004 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.949005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:21.949140 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:21.949715 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:21.949724 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:21.951250 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:21.953131 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:21.953138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:21.953141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:21.953441 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:21.953658 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:21.953658 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:21.953659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:21.953763 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:21.954059 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:21.954063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:21.954170 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 22:48:21.954253 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 22:48:21.954366 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260926 22:48:21.954533 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:21.945484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:44563 (local address 127.14.184.254:34585)
0926 22:48:21.945545 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:21.945548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:21.945707 (+ 159us) server_negotiation.cc:407] Connection header received
0926 22:48:21.946156 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:21.946157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:21.946166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:21.946223 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:21.947145 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.947634 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.948754 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:21.948943 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:21.948994 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:21.951348 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:21.951349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:21.951350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:21.951358 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:21.952990 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:21.953524 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:21.953525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:21.953525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:21.953587 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:21.953831 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:21.953833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:21.953833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:21.953989 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:21.954069 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:21.954302 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 22:48:21.954395 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 22:48:21.955135 2084 heartbeater.cc:347] Connected to a master server at 127.14.184.254:34585
I20260926 22:48:21.955230 2084 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:21.955403 2084 heartbeater.cc:511] Master 127.14.184.254:34585 requested a full tablet report, sending...
I20260926 22:48:21.955760 1759 ts_manager.cc:194] Registered new tserver with Master: 5cb222f435b94bb4bfd19bbfde347ec0 (127.14.184.194:37255)
I20260926 22:48:21.956084 1759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.194@KRBTEST.COM'} at 127.14.184.194:44563
WARNING: no policy specified for kudu/127.14.184.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
Entry for principal kudu/127.14.184.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab.
I20260926 22:48:21.973483 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34585
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:22.050622 2089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:22.050871 2089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:22.050909 2089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:22.051986 2089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:22.052049 2089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:22.052093 2089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:48:22.053438 2089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.195.keytab
--principal=kudu/127.14.184.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:22.053681 2089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:22.053887 2089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:22.058552 2097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.058564 2094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.058560 2095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:22.058841 2089 server_base.cc:1034] running on GCE node
I20260926 22:48:22.059074 2089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:22.059360 2089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:22.060528 2089 hybrid_clock.cc:648] HybridClock initialized: now 1790462902060522 us; error 37 us; skew 500 ppm
Sep 26 22:48:22 dist-test-slave-3986 krb5kdc[1714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462902, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:22.063021 2089 init.cc:377] Logged in from keytab as kudu/127.14.184.195@KRBTEST.COM (short username kudu)
I20260926 22:48:22.063421 2089 webserver.cc:460] Webserver started at http://127.14.184.195:39935/ using document root <none> and password file <none>
I20260926 22:48:22.063624 2089 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:22.063691 2089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:22.063829 2089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:22.064800 2089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "394fc3f11c00470b86ca19e4c459487d"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "7b6b719294d5b7b89dd9057654aeb25b"
server_key_iv: "dfdfe4ee72f0daa73a8f152353d559e9"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.065145 2089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "394fc3f11c00470b86ca19e4c459487d"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "7b6b719294d5b7b89dd9057654aeb25b"
server_key_iv: "dfdfe4ee72f0daa73a8f152353d559e9"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.066373 2089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:22.067122 2104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.067286 2089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.067411 2089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "394fc3f11c00470b86ca19e4c459487d"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "7b6b719294d5b7b89dd9057654aeb25b"
server_key_iv: "dfdfe4ee72f0daa73a8f152353d559e9"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.067466 2089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:22.080339 2089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:22.082464 2089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:22.082566 2089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:22.082829 2089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:22.083056 2089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:22.083086 2089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.083104 2089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:22.083117 2089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.088923 2089 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:35467
I20260926 22:48:22.089010 2217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:35467 every 8 connection(s)
I20260926 22:48:22.089305 2089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
Sep 26 22:48:22 dist-test-slave-3986 krb5kdc[1714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462902, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.195@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:22.097868 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2089
I20260926 22:48:22.097995 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:48:22.098237 15075 external_mini_cluster.cc:1428] Setting key 51415bb8beff9d92b7f32f5c7e849871
I20260926 22:48:22.098752 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.090117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:33773 (local address 127.14.184.254:34585)
0926 22:48:22.090186 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.090188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.090386 (+ 198us) server_negotiation.cc:407] Connection header received
0926 22:48:22.090930 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.090931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.090942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.091000 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.091906 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.092420 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.093358 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.093543 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.093594 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.095650 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.095651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.095651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:22.095658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.097101 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:22.097539 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:22.097540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:22.097541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:22.097596 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:22.097916 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:22.097917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:22.097917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:22.098096 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.098166 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.098515 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.098621 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 22:48:22.098956 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.090085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34585 (local address 127.14.184.195:33773)
0926 22:48:22.090255 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.090297 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.090841 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.091056 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.091059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.091626 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:22.091818 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.091822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.092499 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.092500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.093287 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.093289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.093614 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.093614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.093710 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.094167 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:22.094176 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:22.095552 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:22.097207 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:22.097214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:22.097216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:22.097436 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:22.097669 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:22.097670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:22.097671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:22.097843 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:22.098174 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:22.098182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:22.098351 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.098469 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.098649 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260926 22:48:22.099349 2218 heartbeater.cc:347] Connected to a master server at 127.14.184.254:34585
I20260926 22:48:22.099468 2218 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:22.099653 2218 heartbeater.cc:511] Master 127.14.184.254:34585 requested a full tablet report, sending...
I20260926 22:48:22.100016 1759 ts_manager.cc:194] Registered new tserver with Master: 394fc3f11c00470b86ca19e4c459487d (127.14.184.195:35467)
I20260926 22:48:22.100440 1759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.184.195@KRBTEST.COM'} at 127.14.184.195:33773
I20260926 22:48:22.110270 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:48:22.117650 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.111791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.14.184.254:34585)
0926 22:48:22.111876 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.111877 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.111912 (+ 35us) server_negotiation.cc:407] Connection header received
0926 22:48:22.111978 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.111979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.111987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.112033 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.112567 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.113039 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.113797 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.113966 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.114016 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.115319 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.115320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.115320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:22.115325 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.116612 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:22.116941 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:22.116942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:22.116942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:22.116988 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:22.117195 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:22.117195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:22.117196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:22.117317 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.117391 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.117503 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.117560 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 22:48:22.119401 1759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:48:22.120906 1759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:48:22.127498 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.123288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:35467 (local address 127.0.0.1:38044)
0926 22:48:22.123856 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.123866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.123911 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.124181 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.124182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.124184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.124216 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.124218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.125499 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.125500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.126475 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.126478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.127200 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.127201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.127286 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.127299 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.127368 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.127424 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":37,"threads_started":1}
I20260926 22:48:22.127800 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.123083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:38429 (local address 127.0.0.1:36992)
0926 22:48:22.123367 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.123379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.123431 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.123749 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.123752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.123754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.123848 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.123850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.125213 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.125214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.126361 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.126364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.127555 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.127556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.127629 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.127633 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.127693 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.127736 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":36,"thread_start_us":52,"threads_started":1}
I20260926 22:48:22.127811 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.123397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.14.184.195:35467)
0926 22:48:22.123544 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.123549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.123976 (+ 427us) server_negotiation.cc:407] Connection header received
0926 22:48:22.124020 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.124022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.124037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.124104 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.124282 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.125389 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.126549 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.127109 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.127201 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.127258 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.127694 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.127735 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260926 22:48:22.127825 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.123415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.14.184.193:38429)
0926 22:48:22.123577 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.123581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.123590 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:22.123628 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.123631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.123650 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.123746 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.123910 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.125102 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.126743 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.127459 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.127555 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.127609 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.127707 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.127744 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260926 22:48:22.127929 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.123081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:37255 (local address 127.0.0.1:53746)
0926 22:48:22.123245 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.123291 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.123357 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.123765 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.123767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.123769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.123857 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.123859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.125054 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.125055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.126362 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.126364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.127324 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.127324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.127389 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.127391 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.127448 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.127481 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":10,"thread_start_us":54,"threads_started":1}
I20260926 22:48:22.127825 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.123174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.14.184.194:37255)
0926 22:48:22.123594 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.123597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.123606 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:22.123648 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.123649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.123661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.123764 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.123981 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.124782 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.126450 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.127212 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.127321 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.127373 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.127464 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.127513 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":97,"threads_started":1}
I20260926 22:48:22.128589 1884 tablet_service.cc:1467] Processing CreateTablet for tablet bdd05c9cc75b483688e3fd2afa46ce82 (DEFAULT_TABLE table=test-table [id=aaa3e3a9ea5a4575aa214fef282e5db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:22.128906 1884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd05c9cc75b483688e3fd2afa46ce82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.128979 2018 tablet_service.cc:1467] Processing CreateTablet for tablet bdd05c9cc75b483688e3fd2afa46ce82 (DEFAULT_TABLE table=test-table [id=aaa3e3a9ea5a4575aa214fef282e5db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:22.129318 2018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd05c9cc75b483688e3fd2afa46ce82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.131156 2152 tablet_service.cc:1467] Processing CreateTablet for tablet bdd05c9cc75b483688e3fd2afa46ce82 (DEFAULT_TABLE table=test-table [id=aaa3e3a9ea5a4575aa214fef282e5db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:22.131490 2152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd05c9cc75b483688e3fd2afa46ce82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.131678 2232 tablet_bootstrap.cc:492] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754: Bootstrap starting.
I20260926 22:48:22.131944 2233 tablet_bootstrap.cc:492] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Bootstrap starting.
I20260926 22:48:22.132335 2232 tablet_bootstrap.cc:654] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.132680 2233 tablet_bootstrap.cc:654] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.132890 2232 log.cc:824] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.133271 2233 log.cc:824] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.133862 2232 tablet_bootstrap.cc:492] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754: No bootstrap required, opened a new log
I20260926 22:48:22.133946 2232 ts_tablet_manager.cc:1397] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:22.134007 2233 tablet_bootstrap.cc:492] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: No bootstrap required, opened a new log
I20260926 22:48:22.134074 2233 ts_tablet_manager.cc:1397] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:22.134953 2236 tablet_bootstrap.cc:492] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Bootstrap starting.
I20260926 22:48:22.135658 2236 tablet_bootstrap.cc:654] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.135865 2233 raft_consensus.cc:348] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.136021 2233 raft_consensus.cc:374] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.136061 2233 raft_consensus.cc:729] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb222f435b94bb4bfd19bbfde347ec0, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.135914 2232 raft_consensus.cc:348] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.136113 2232 raft_consensus.cc:374] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.136158 2232 raft_consensus.cc:729] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acdd65e5b44b4323b0ac880883e75754, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.136178 2233 consensus_queue.cc:260] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.136312 2236 log.cc:824] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.136319 2232 consensus_queue.cc:260] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.136489 2233 ts_tablet_manager.cc:1428] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 22:48:22.136534 2084 heartbeater.cc:503] Master 127.14.184.254:34585 was elected leader, sending a full tablet report...
I20260926 22:48:22.136600 2232 ts_tablet_manager.cc:1428] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:48:22.136725 1950 heartbeater.cc:503] Master 127.14.184.254:34585 was elected leader, sending a full tablet report...
I20260926 22:48:22.137351 2236 tablet_bootstrap.cc:492] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: No bootstrap required, opened a new log
I20260926 22:48:22.137415 2236 ts_tablet_manager.cc:1397] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 22:48:22.138960 2236 raft_consensus.cc:348] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.139092 2236 raft_consensus.cc:374] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.139114 2236 raft_consensus.cc:729] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394fc3f11c00470b86ca19e4c459487d, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.139231 2236 consensus_queue.cc:260] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.139504 2236 ts_tablet_manager.cc:1428] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 22:48:22.139537 2218 heartbeater.cc:503] Master 127.14.184.254:34585 was elected leader, sending a full tablet report...
I20260926 22:48:22.178829 2238 raft_consensus.cc:482] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:22.178988 2238 raft_consensus.cc:504] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.179895 2238 leader_election.cc:288] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cb222f435b94bb4bfd19bbfde347ec0 (127.14.184.194:37255), 394fc3f11c00470b86ca19e4c459487d (127.14.184.195:35467)
I20260926 22:48:22.182497 1952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.179557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:37255 (local address 127.14.184.193:46203)
0926 22:48:22.179643 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.179655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.179698 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.179943 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.179944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.179946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.180022 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.180025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.180651 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.180653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.181522 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.181525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.182189 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.182190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.182303 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.182307 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.182370 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.182418 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":4}
I20260926 22:48:22.182511 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.179670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:46203 (local address 127.14.184.194:37255)
0926 22:48:22.179793 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.179795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.179803 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:48:22.179837 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.179838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.179851 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.179926 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.180116 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.180574 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.181608 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.182112 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.182166 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.182172 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.182401 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.182439 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 22:48:22.182709 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.179710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:35467 (local address 127.14.184.193:34443)
0926 22:48:22.179893 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.179904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.179983 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.180137 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.180137 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.180138 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.180221 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.180223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.180989 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.180990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.181779 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.181781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.182429 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.182430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.182499 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.182502 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.182562 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.182602 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260926 22:48:22.182838 2038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd05c9cc75b483688e3fd2afa46ce82" candidate_uuid: "acdd65e5b44b4323b0ac880883e75754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" is_pre_election: true
I20260926 22:48:22.182971 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.179832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:34443 (local address 127.14.184.195:35467)
0926 22:48:22.179871 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.179873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.179956 (+ 83us) server_negotiation.cc:407] Connection header received
0926 22:48:22.180067 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.180068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.180078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.180121 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.180300 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.180896 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.181880 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.182351 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.182413 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.182420 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.182842 (+ 422us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.182886 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:22.182981 2038 raft_consensus.cc:2454] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acdd65e5b44b4323b0ac880883e75754 in term 0.
I20260926 22:48:22.183194 1839 leader_election.cc:302] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb222f435b94bb4bfd19bbfde347ec0, acdd65e5b44b4323b0ac880883e75754; no voters:
I20260926 22:48:22.183336 2238 raft_consensus.cc:2781] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:48:22.183346 2172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd05c9cc75b483688e3fd2afa46ce82" candidate_uuid: "acdd65e5b44b4323b0ac880883e75754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394fc3f11c00470b86ca19e4c459487d" is_pre_election: true
I20260926 22:48:22.183413 2238 raft_consensus.cc:482] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:22.183445 2238 raft_consensus.cc:3037] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.183485 2172 raft_consensus.cc:2454] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acdd65e5b44b4323b0ac880883e75754 in term 0.
I20260926 22:48:22.184276 2238 raft_consensus.cc:504] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.184418 2238 leader_election.cc:288] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [CANDIDATE]: Term 1 election: Requested vote from peers 5cb222f435b94bb4bfd19bbfde347ec0 (127.14.184.194:37255), 394fc3f11c00470b86ca19e4c459487d (127.14.184.195:35467)
I20260926 22:48:22.184572 2038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd05c9cc75b483688e3fd2afa46ce82" candidate_uuid: "acdd65e5b44b4323b0ac880883e75754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb222f435b94bb4bfd19bbfde347ec0"
I20260926 22:48:22.184607 2172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd05c9cc75b483688e3fd2afa46ce82" candidate_uuid: "acdd65e5b44b4323b0ac880883e75754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394fc3f11c00470b86ca19e4c459487d"
I20260926 22:48:22.184662 2038 raft_consensus.cc:3037] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.184684 2172 raft_consensus.cc:3037] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.185344 2172 raft_consensus.cc:2454] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acdd65e5b44b4323b0ac880883e75754 in term 1.
I20260926 22:48:22.185428 2038 raft_consensus.cc:2454] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acdd65e5b44b4323b0ac880883e75754 in term 1.
I20260926 22:48:22.185518 1839 leader_election.cc:302] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394fc3f11c00470b86ca19e4c459487d, acdd65e5b44b4323b0ac880883e75754; no voters:
I20260926 22:48:22.185616 2238 raft_consensus.cc:2781] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:22.185760 2238 raft_consensus.cc:686] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [term 1 LEADER]: Becoming Leader. State: Replica: acdd65e5b44b4323b0ac880883e75754, State: Running, Role: LEADER
I20260926 22:48:22.185863 2238 consensus_queue.cc:237] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } }
I20260926 22:48:22.186625 1758 catalog_manager.cc:5495] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 reported cstate change: term changed from 0 to 1, leader changed from <none> to acdd65e5b44b4323b0ac880883e75754 (127.14.184.193). New cstate: current_term: 1 leader_uuid: "acdd65e5b44b4323b0ac880883e75754" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acdd65e5b44b4323b0ac880883e75754" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 } health_report { overall_health: UNKNOWN } } }
W20260926 22:48:22.195264 2085 tablet.cc:2369] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 22:48:22.201977 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.199780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.14.184.193:38429)
0926 22:48:22.199825 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.199827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.199851 (+ 24us) server_negotiation.cc:407] Connection header received
0926 22:48:22.199926 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.199927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.199945 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.200006 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:22.200114 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.200679 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.201376 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.201555 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.201603 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.201716 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:22.201808 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:22.201863 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.201895 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.201910 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:22.203673 2172 raft_consensus.cc:1261] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 1 FOLLOWER]: Refusing update from remote peer acdd65e5b44b4323b0ac880883e75754: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:22.203722 2038 raft_consensus.cc:1261] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 1 FOLLOWER]: Refusing update from remote peer acdd65e5b44b4323b0ac880883e75754: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:22.204000 2238 consensus_queue.cc:1035] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [LEADER]: Connected to new peer: Peer: permanent_uuid: "394fc3f11c00470b86ca19e4c459487d" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 35467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:22.204119 2242 consensus_queue.cc:1035] T bdd05c9cc75b483688e3fd2afa46ce82 P acdd65e5b44b4323b0ac880883e75754 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb222f435b94bb4bfd19bbfde347ec0" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 37255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:22.206471 2245 mvcc.cc:205] Tried to move back new op lower bound from 7333736047423848448 to 7333736047353937920. Current Snapshot: MvccSnapshot[applied={T|T < 7333736047423848448}]
I20260926 22:48:22.206496 2247 mvcc.cc:205] Tried to move back new op lower bound from 7333736047423848448 to 7333736047353937920. Current Snapshot: MvccSnapshot[applied={T|T < 7333736047423848448}]
I20260926 22:48:22.206652 2246 mvcc.cc:205] Tried to move back new op lower bound from 7333736047423848448 to 7333736047353937920. Current Snapshot: MvccSnapshot[applied={T|T < 7333736047423848448}]
I20260926 22:48:22.208657 1759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:22.208752 1759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:22.209403 1759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e19e65da2c8428098868f6160bfc0b0: Sending DeleteTablet for 3 replicas of tablet bdd05c9cc75b483688e3fd2afa46ce82
I20260926 22:48:22.209724 2018 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd05c9cc75b483688e3fd2afa46ce82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:22 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:53746
I20260926 22:48:22.209745 1884 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd05c9cc75b483688e3fd2afa46ce82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:22 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:36992
I20260926 22:48:22.209962 2255 tablet_replica.cc:330] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: stopping tablet replica
I20260926 22:48:22.209969 2152 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd05c9cc75b483688e3fd2afa46ce82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:22 UTC) from {username='kudu', principal='kudu/127.14.184.254@KRBTEST.COM'} at 127.0.0.1:38044
I20260926 22:48:22.210065 2255 raft_consensus.cc:2229] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:22.210223 2255 raft_consensus.cc:2258] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:22.210225 2257 tablet_replica.cc:330] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: stopping tablet replica
I20260926 22:48:22.210391 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1821
I20260926 22:48:22.211186 2257 raft_consensus.cc:2229] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:22.211293 2257 raft_consensus.cc:2258] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:22.212293 2255 ts_tablet_manager.cc:1905] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:22.212384 2257 ts_tablet_manager.cc:1905] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:22.214138 2255 ts_tablet_manager.cc:1918] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:22.214138 2257 ts_tablet_manager.cc:1918] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:22.214214 2255 log.cc:1196] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/bdd05c9cc75b483688e3fd2afa46ce82
I20260926 22:48:22.214226 2257 log.cc:1196] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/bdd05c9cc75b483688e3fd2afa46ce82
I20260926 22:48:22.214496 2255 ts_tablet_manager.cc:1939] T bdd05c9cc75b483688e3fd2afa46ce82 P 5cb222f435b94bb4bfd19bbfde347ec0: Deleting consensus metadata
I20260926 22:48:22.214496 2257 ts_tablet_manager.cc:1939] T bdd05c9cc75b483688e3fd2afa46ce82 P 394fc3f11c00470b86ca19e4c459487d: Deleting consensus metadata
I20260926 22:48:22.214951 1745 catalog_manager.cc:4841] TS 5cb222f435b94bb4bfd19bbfde347ec0 (127.14.184.194:37255): tablet bdd05c9cc75b483688e3fd2afa46ce82 (table test-table [id=aaa3e3a9ea5a4575aa214fef282e5db4]) successfully deleted
I20260926 22:48:22.215046 1745 catalog_manager.cc:4841] TS 394fc3f11c00470b86ca19e4c459487d (127.14.184.195:35467): tablet bdd05c9cc75b483688e3fd2afa46ce82 (table test-table [id=aaa3e3a9ea5a4575aa214fef282e5db4]) successfully deleted
W20260926 22:48:22.215615 1746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:38429 (error 108)
I20260926 22:48:22.215883 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1955
I20260926 22:48:22.216109 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.215986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:38429 (local address 127.0.0.1:37008)
0926 22:48:22.216032 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.216070 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:38429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:48:22.216210 1746 catalog_manager.cc:4584] TS acdd65e5b44b4323b0ac880883e75754 (127.14.184.193:38429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bdd05c9cc75b483688e3fd2afa46ce82: Network error: Client connection negotiation failed: client connection to 127.14.184.193:38429: connect: Connection refused (error 111)
I20260926 22:48:22.220887 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2089
I20260926 22:48:22.225356 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 1727
2026-09-26T22:48:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1875 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T22:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:22Z Disabled control of system clock
I20260926 22:48:22.252631 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33049
--webserver_interface=127.14.184.254
--webserver_port=0
--builtin_ntp_servers=127.14.184.212:45801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:33049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:22.330426 2262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:22.330627 2262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:22.330660 2262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:22.331792 2262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:22.331835 2262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:22.331846 2262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:22.331858 2262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:22.331871 2262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:22.333127 2262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:33049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:22.333310 2262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:22.333524 2262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:22.338215 2267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.338308 2270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.338344 2268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:22.338420 2262 server_base.cc:1034] running on GCE node
I20260926 22:48:22.338762 2262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:22.339056 2262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:22.340198 2262 hybrid_clock.cc:648] HybridClock initialized: now 1790462902340183 us; error 29 us; skew 500 ppm
I20260926 22:48:22.341444 2262 webserver.cc:460] Webserver started at http://127.14.184.254:33817/ using document root <none> and password file <none>
I20260926 22:48:22.341712 2262 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:22.341761 2262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:22.341979 2262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:22.343091 2262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "b374722ffb1941a4b97a2785e22bb826"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "15e3510fde14648eafadf7e8bbdf25c7"
server_key_iv: "16723bb6fdce5b9dfadee93585ccfec5"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.343426 2262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "b374722ffb1941a4b97a2785e22bb826"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "15e3510fde14648eafadf7e8bbdf25c7"
server_key_iv: "16723bb6fdce5b9dfadee93585ccfec5"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.344620 2262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:22.345263 2276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.345427 2262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:22.345521 2262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "b374722ffb1941a4b97a2785e22bb826"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "15e3510fde14648eafadf7e8bbdf25c7"
server_key_iv: "16723bb6fdce5b9dfadee93585ccfec5"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.345598 2262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:22.352542 2262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:22.352809 2262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:22.352954 2262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:22.356739 2262 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:33049
I20260926 22:48:22.356776 2328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:33049 every 8 connection(s)
I20260926 22:48:22.357079 2262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:22.357999 2329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.360199 2329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826: Bootstrap starting.
I20260926 22:48:22.360746 2329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.361049 2329 log.cc:824] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.361722 2329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826: No bootstrap required, opened a new log
I20260926 22:48:22.363289 2329 raft_consensus.cc:348] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b374722ffb1941a4b97a2785e22bb826" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33049 } }
I20260926 22:48:22.363407 2329 raft_consensus.cc:374] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.363428 2329 raft_consensus.cc:729] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b374722ffb1941a4b97a2785e22bb826, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.363603 2329 consensus_queue.cc:260] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b374722ffb1941a4b97a2785e22bb826" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33049 } }
I20260926 22:48:22.363662 2329 raft_consensus.cc:388] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:22.363683 2329 raft_consensus.cc:482] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:22.363744 2329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.364226 2329 raft_consensus.cc:504] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b374722ffb1941a4b97a2785e22bb826" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33049 } }
I20260926 22:48:22.364331 2329 leader_election.cc:302] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b374722ffb1941a4b97a2785e22bb826; no voters:
I20260926 22:48:22.364518 2329 leader_election.cc:288] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:22.364595 2332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:22.364761 2332 raft_consensus.cc:686] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [term 1 LEADER]: Becoming Leader. State: Replica: b374722ffb1941a4b97a2785e22bb826, State: Running, Role: LEADER
I20260926 22:48:22.364779 2329 sys_catalog.cc:564] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:22.364880 2332 consensus_queue.cc:237] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b374722ffb1941a4b97a2785e22bb826" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33049 } }
I20260926 22:48:22.365389 2334 sys_catalog.cc:455] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b374722ffb1941a4b97a2785e22bb826. Latest consensus state: current_term: 1 leader_uuid: "b374722ffb1941a4b97a2785e22bb826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b374722ffb1941a4b97a2785e22bb826" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33049 } } }
I20260926 22:48:22.365490 2334 sys_catalog.cc:458] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:22.365641 2333 sys_catalog.cc:455] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b374722ffb1941a4b97a2785e22bb826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b374722ffb1941a4b97a2785e22bb826" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33049 } } }
I20260926 22:48:22.365688 2333 sys_catalog.cc:458] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:22.365751 2341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:22.366252 2341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:22.366494 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2262
I20260926 22:48:22.366603 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:22.366828 15075 external_mini_cluster.cc:1428] Setting key 3fc97b25f43e4ea48587ddc291f50fed
I20260926 22:48:22.367947 2341 catalog_manager.cc:1347] Generated new cluster ID: e1041b29105248c3a0c8d6afb970a1ee
I20260926 22:48:22.367998 2341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:22.372888 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.367311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.14.184.254:33049)
0926 22:48:22.367495 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.367504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.367561 (+ 57us) server_negotiation.cc:407] Connection header received
0926 22:48:22.367720 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.367728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.367794 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.367939 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.368251 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.369577 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.370689 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.371280 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.371386 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.372062 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.372066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.372066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:22.372261 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.372300 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:22.372312 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.372371 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.372459 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.372528 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260926 22:48:22.383529 2341 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:22.384173 2341 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:22.397359 2341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826: Generated new TSK 0
I20260926 22:48:22.397547 2341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:22.404933 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.193:0
--local_ip_for_outbound_sockets=127.14.184.193
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33049
--builtin_ntp_servers=127.14.184.212:45801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:22.489741 2353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:22.489961 2353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:22.489998 2353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:22.491197 2353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:22.491261 2353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:22.491303 2353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.193
I20260926 22:48:22.492625 2353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.184.193
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.193
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:22.492882 2353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:22.493196 2353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:22.498935 2361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.498943 2359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.499017 2358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:22.498962 2353 server_base.cc:1034] running on GCE node
I20260926 22:48:22.499338 2353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:22.499593 2353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:22.500744 2353 hybrid_clock.cc:648] HybridClock initialized: now 1790462902500727 us; error 37 us; skew 500 ppm
I20260926 22:48:22.501958 2353 webserver.cc:460] Webserver started at http://127.14.184.193:36341/ using document root <none> and password file <none>
I20260926 22:48:22.502188 2353 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:22.502238 2353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:22.502400 2353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:22.503455 2353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data/instance:
uuid: "8af71f4ba6a947978505822137a49dcc"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "50cfcb74d04ec7b8d3303e66bd19f561"
server_key_iv: "420df4f43dceb196c970c0c3aa383925"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.503794 2353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance:
uuid: "8af71f4ba6a947978505822137a49dcc"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "50cfcb74d04ec7b8d3303e66bd19f561"
server_key_iv: "420df4f43dceb196c970c0c3aa383925"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.505007 2353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 22:48:22.505683 2367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.505859 2353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:22.505942 2353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal
uuid: "8af71f4ba6a947978505822137a49dcc"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "50cfcb74d04ec7b8d3303e66bd19f561"
server_key_iv: "420df4f43dceb196c970c0c3aa383925"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.505992 2353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:22.533717 2353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:22.534001 2353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:22.534101 2353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:22.534286 2353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:22.534579 2353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:22.534627 2353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.534705 2353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:22.534752 2353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.540485 2353 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.193:35315
I20260926 22:48:22.540570 2480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.193:35315 every 8 connection(s)
I20260926 22:48:22.540886 2353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/data/info.pb
I20260926 22:48:22.545825 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.541609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.193:58137 (local address 127.14.184.254:33049)
0926 22:48:22.541699 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.541701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.541927 (+ 226us) server_negotiation.cc:407] Connection header received
0926 22:48:22.542382 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.542383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.542395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.542448 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.543166 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.543640 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.544569 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.544752 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.544802 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.545606 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.545607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.545607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:22.545611 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.545622 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:22.545625 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.545665 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.545748 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.545768 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:48:22.546047 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.541588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33049 (local address 127.14.184.193:58137)
0926 22:48:22.541797 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.541838 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.542302 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.542512 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.542515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.542523 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:22.543092 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.543098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.543731 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.543732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.544496 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.544498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.544827 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.544828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.544935 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.545486 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:22.545486 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:22.545514 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:22.545535 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:22.545674 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:22.545676 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:22.545678 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.545731 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.545808 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
I20260926 22:48:22.546622 2481 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33049
I20260926 22:48:22.546744 2481 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:22.546916 2481 heartbeater.cc:511] Master 127.14.184.254:33049 requested a full tablet report, sending...
I20260926 22:48:22.547408 2293 ts_manager.cc:194] Registered new tserver with Master: 8af71f4ba6a947978505822137a49dcc (127.14.184.193:35315)
I20260926 22:48:22.548059 2293 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.193:58137
I20260926 22:48:22.550058 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2353
I20260926 22:48:22.550160 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-0/wal/instance
I20260926 22:48:22.550361 15075 external_mini_cluster.cc:1428] Setting key 7ae5e15efa64ed92f91a144c9733df4b
I20260926 22:48:22.551275 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.194:0
--local_ip_for_outbound_sockets=127.14.184.194
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33049
--builtin_ntp_servers=127.14.184.212:45801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 22:48:22.630183 2484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:22.630370 2484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:22.630391 2484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:22.631459 2484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:22.631505 2484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:22.631527 2484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.194
I20260926 22:48:22.632720 2484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.184.194
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.194
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:22.632879 2484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:22.633083 2484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:22.637952 2490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.637964 2492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.637959 2489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:22.638181 2484 server_base.cc:1034] running on GCE node
I20260926 22:48:22.638405 2484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:22.638603 2484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:22.639744 2484 hybrid_clock.cc:648] HybridClock initialized: now 1790462902639731 us; error 42 us; skew 500 ppm
I20260926 22:48:22.640944 2484 webserver.cc:460] Webserver started at http://127.14.184.194:33883/ using document root <none> and password file <none>
I20260926 22:48:22.641183 2484 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:22.641232 2484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:22.641394 2484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:22.642402 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data/instance:
uuid: "052554ad43344a74be5118f625d84433"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "539835ed853b65b05e80c7b714c87f4e"
server_key_iv: "82d0f26004bbe7eea2d2044fff3a4120"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.642802 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance:
uuid: "052554ad43344a74be5118f625d84433"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "539835ed853b65b05e80c7b714c87f4e"
server_key_iv: "82d0f26004bbe7eea2d2044fff3a4120"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.644011 2484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:22.644668 2498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.644862 2484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:22.644944 2484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal
uuid: "052554ad43344a74be5118f625d84433"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "539835ed853b65b05e80c7b714c87f4e"
server_key_iv: "82d0f26004bbe7eea2d2044fff3a4120"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.645009 2484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:22.655956 2484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:22.656216 2484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:22.656353 2484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:22.656554 2484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:22.656867 2484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:22.656899 2484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.656919 2484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:22.656965 2484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.662580 2484 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.194:39925
I20260926 22:48:22.662633 2611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.194:39925 every 8 connection(s)
I20260926 22:48:22.663009 2484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/data/info.pb
I20260926 22:48:22.665491 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2484
I20260926 22:48:22.665607 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal/instance
I20260926 22:48:22.665858 15075 external_mini_cluster.cc:1428] Setting key 79b21fc7af114f9a74aaed9d3ee25564
I20260926 22:48:22.666872 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.184.195:0
--local_ip_for_outbound_sockets=127.14.184.195
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33049
--builtin_ntp_servers=127.14.184.212:45801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 22:48:22.669083 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.663739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:53159 (local address 127.14.184.254:33049)
0926 22:48:22.663809 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.663811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.664024 (+ 213us) server_negotiation.cc:407] Connection header received
0926 22:48:22.664466 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.664468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.664477 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.664539 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.665239 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.666016 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.667528 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.667712 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.667773 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.668831 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.668833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.668833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:22.668838 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.668853 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:22.668857 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.668905 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.668979 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.669007 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:48:22.669356 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.663730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33049 (local address 127.14.184.194:53159)
0926 22:48:22.663899 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.663937 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.664385 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.664589 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.664592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.664600 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:22.665153 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.665158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.666141 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.666142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.667431 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.667434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.667802 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.667803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.667961 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.668693 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:22.668693 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:22.668721 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:22.668742 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:22.668905 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:22.668909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:22.668912 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.668981 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.669077 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260926 22:48:22.669834 2612 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33049
I20260926 22:48:22.669929 2612 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:22.670110 2612 heartbeater.cc:511] Master 127.14.184.254:33049 requested a full tablet report, sending...
I20260926 22:48:22.670488 2293 ts_manager.cc:194] Registered new tserver with Master: 052554ad43344a74be5118f625d84433 (127.14.184.194:39925)
I20260926 22:48:22.671065 2293 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.194:53159
W20260926 22:48:22.746807 2615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:22.747001 2615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:22.747023 2615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:22.748344 2615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:22.748415 2615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:22.748456 2615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.184.195
I20260926 22:48:22.749863 2615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.184.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.184.195
--webserver_port=0
--tserver_master_addrs=127.14.184.254:33049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.184.195
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:22.750121 2615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:22.750411 2615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:22.755482 2621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.755513 2623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:22.755566 2620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:22.755594 2615 server_base.cc:1034] running on GCE node
I20260926 22:48:22.755891 2615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:22.756155 2615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:22.757321 2615 hybrid_clock.cc:648] HybridClock initialized: now 1790462902757310 us; error 38 us; skew 500 ppm
I20260926 22:48:22.758579 2615 webserver.cc:460] Webserver started at http://127.14.184.195:34503/ using document root <none> and password file <none>
I20260926 22:48:22.758844 2615 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:22.758915 2615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:22.759050 2615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:22.760104 2615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data/instance:
uuid: "e96b081572364dfbbc5d1671b2be0b86"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "105d5c03aed42c97879185b2175195d5"
server_key_iv: "e096ee46e0e0fe4590bf68d74bd2a22b"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.760473 2615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance:
uuid: "e96b081572364dfbbc5d1671b2be0b86"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "105d5c03aed42c97879185b2175195d5"
server_key_iv: "e096ee46e0e0fe4590bf68d74bd2a22b"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.761745 2615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:22.762485 2629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.762660 2615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:22.762785 2615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal
uuid: "e96b081572364dfbbc5d1671b2be0b86"
format_stamp: "Formatted at 2026-09-26 22:48:22 on dist-test-slave-3986"
server_key: "105d5c03aed42c97879185b2175195d5"
server_key_iv: "e096ee46e0e0fe4590bf68d74bd2a22b"
server_key_version: "encryptionkey@0"
I20260926 22:48:22.762873 2615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:22.773891 2615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:22.774176 2615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:22.774330 2615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:22.774549 2615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 22:48:22.774909 2615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 22:48:22.774964 2615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.775007 2615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 22:48:22.775032 2615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:22.780946 2615 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.195:37981
I20260926 22:48:22.780982 2742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.195:37981 every 8 connection(s)
I20260926 22:48:22.781317 2615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/data/info.pb
I20260926 22:48:22.781996 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2615
I20260926 22:48:22.782110 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal/instance
I20260926 22:48:22.782313 15075 external_mini_cluster.cc:1428] Setting key 3a77762984fe06bdadbbaf983d7bbfff
I20260926 22:48:22.786348 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.782080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.195:52627 (local address 127.14.184.254:33049)
0926 22:48:22.782486 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.782488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.782498 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:22.783001 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.783002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.783011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.783045 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.783727 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.784209 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.785101 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.785283 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.785332 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.786138 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.786139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.786139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:22.786143 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.786152 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:22.786155 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.786190 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.786273 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.786293 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260926 22:48:22.786530 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.782040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:33049 (local address 127.14.184.195:52627)
0926 22:48:22.782211 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.782258 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.782914 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.783131 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.783135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.783145 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:22.783655 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.783660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.784286 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.784287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.785032 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.785034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.785352 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.785352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.785456 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.786006 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 22:48:22.786006 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:22.786039 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 22:48:22.786060 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:22.786192 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:22.786194 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:22.786196 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.786267 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.786338 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260926 22:48:22.786983 2743 heartbeater.cc:347] Connected to a master server at 127.14.184.254:33049
I20260926 22:48:22.787098 2743 heartbeater.cc:464] Registering TS with master...
I20260926 22:48:22.787307 2743 heartbeater.cc:511] Master 127.14.184.254:33049 requested a full tablet report, sending...
I20260926 22:48:22.787643 2293 ts_manager.cc:194] Registered new tserver with Master: e96b081572364dfbbc5d1671b2be0b86 (127.14.184.195:37981)
I20260926 22:48:22.788028 2293 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.184.195:52627
I20260926 22:48:22.794047 15075 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 22:48:22.798424 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.795537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.14.184.254:33049)
0926 22:48:22.795635 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.795638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.795739 (+ 101us) server_negotiation.cc:407] Connection header received
0926 22:48:22.795816 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.795817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.795829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.795879 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:22.796044 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.796519 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.797280 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.797483 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.797558 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.798167 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:22.798168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:22.798169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:22.798173 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:22.798185 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:22.798189 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:22.798261 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.798314 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.798338 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 22:48:22.799929 2293 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 22:48:22.801450 2293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 22:48:22.807580 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.803795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.14.184.195:37981)
0926 22:48:22.804066 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.804069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.804078 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:22.804108 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.804110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.804143 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.804219 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.804373 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.805119 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.806533 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.807156 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.807239 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.807327 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.807462 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.807498 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":81,"threads_started":1}
I20260926 22:48:22.807570 2752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.803639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:37981 (local address 127.0.0.1:37646)
0926 22:48:22.803973 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.803984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.804036 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.804279 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.804279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.804281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.804315 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.804316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.805201 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.805202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.806451 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.806454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.807258 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.807259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.807333 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.807346 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.807447 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.807499 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260926 22:48:22.807934 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.803649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.194:39925 (local address 127.0.0.1:46856)
0926 22:48:22.804115 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.804124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.804160 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.804511 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.804512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.804514 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.804551 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.804553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.805710 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.805711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.806580 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.806582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.807595 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.807596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.807651 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.807655 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.807819 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.807863 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":99,"threads_started":1}
I20260926 22:48:22.808167 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.803873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.14.184.194:39925)
0926 22:48:22.804279 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.804282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.804312 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:48:22.804349 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.804351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.804366 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.804437 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.804638 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.805648 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.806644 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.807404 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.807480 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.807558 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.808047 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.808086 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":77,"threads_started":1}
I20260926 22:48:22.808377 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.803814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.14.184.193:35315)
0926 22:48:22.803985 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.803990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.804002 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:48:22.804043 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.804046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.804062 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.804150 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.804448 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.805552 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.806838 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.807537 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.807600 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.808029 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.808260 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.808303 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260926 22:48:22.808574 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.803636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:35315 (local address 127.0.0.1:46358)
0926 22:48:22.803805 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.803842 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.803906 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.804171 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.804174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.804176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.804251 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.804253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.805643 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.805644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.806749 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.806751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.808065 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.808066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.808128 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.808132 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.808468 (+ 336us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.808516 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":33,"thread_start_us":64,"threads_started":1}
I20260926 22:48:22.808595 2677 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 (DEFAULT_TABLE table=test-table [id=bcc12db312194e32b9c19d1acb163c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:22.809033 2677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.809286 2415 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 (DEFAULT_TABLE table=test-table [id=bcc12db312194e32b9c19d1acb163c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:22.809607 2415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.812484 2757 tablet_bootstrap.cc:492] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Bootstrap starting.
I20260926 22:48:22.812824 2758 tablet_bootstrap.cc:492] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc: Bootstrap starting.
I20260926 22:48:22.812825 2546 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 (DEFAULT_TABLE table=test-table [id=bcc12db312194e32b9c19d1acb163c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 22:48:22.813150 2546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:22.813344 2757 tablet_bootstrap.cc:654] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.813426 2758 tablet_bootstrap.cc:654] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.813679 2757 log.cc:824] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.813776 2758 log.cc:824] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.814400 2757 tablet_bootstrap.cc:492] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: No bootstrap required, opened a new log
I20260926 22:48:22.814409 2758 tablet_bootstrap.cc:492] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc: No bootstrap required, opened a new log
I20260926 22:48:22.814476 2757 ts_tablet_manager.cc:1397] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:22.814496 2758 ts_tablet_manager.cc:1397] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:22.816476 2761 tablet_bootstrap.cc:492] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Bootstrap starting.
I20260926 22:48:22.816550 2758 raft_consensus.cc:348] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.816726 2758 raft_consensus.cc:374] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.816768 2758 raft_consensus.cc:729] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af71f4ba6a947978505822137a49dcc, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.816938 2758 consensus_queue.cc:260] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.817222 2758 ts_tablet_manager.cc:1428] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 22:48:22.817246 2761 tablet_bootstrap.cc:654] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:22.817135 2757 raft_consensus.cc:348] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.817251 2481 heartbeater.cc:503] Master 127.14.184.254:33049 was elected leader, sending a full tablet report...
I20260926 22:48:22.817284 2757 raft_consensus.cc:374] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.817317 2757 raft_consensus.cc:729] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96b081572364dfbbc5d1671b2be0b86, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.817418 2757 consensus_queue.cc:260] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.817569 2761 log.cc:824] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:22.817653 2757 ts_tablet_manager.cc:1428] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 22:48:22.817762 2743 heartbeater.cc:503] Master 127.14.184.254:33049 was elected leader, sending a full tablet report...
I20260926 22:48:22.818295 2761 tablet_bootstrap.cc:492] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: No bootstrap required, opened a new log
I20260926 22:48:22.818362 2761 ts_tablet_manager.cc:1397] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 22:48:22.820320 2761 raft_consensus.cc:348] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.820432 2761 raft_consensus.cc:374] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:22.820453 2761 raft_consensus.cc:729] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052554ad43344a74be5118f625d84433, State: Initialized, Role: FOLLOWER
I20260926 22:48:22.820557 2761 consensus_queue.cc:260] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.820760 2761 ts_tablet_manager.cc:1428] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 22:48:22.820838 2612 heartbeater.cc:503] Master 127.14.184.254:33049 was elected leader, sending a full tablet report...
I20260926 22:48:22.881095 2765 raft_consensus.cc:482] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 22:48:22.881265 2765 raft_consensus.cc:504] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.881635 2765 leader_election.cc:288] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8af71f4ba6a947978505822137a49dcc (127.14.184.193:35315), e96b081572364dfbbc5d1671b2be0b86 (127.14.184.195:37981)
I20260926 22:48:22.885701 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.881943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:53231 (local address 127.14.184.193:35315)
0926 22:48:22.882006 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.882008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.882141 (+ 133us) server_negotiation.cc:407] Connection header received
0926 22:48:22.882172 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.882173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.882185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.882241 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.882387 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.883146 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.884661 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.885218 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.885297 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.885305 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.885579 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.885630 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 22:48:22.885706 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.881943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.184.194:59165 (local address 127.14.184.195:37981)
0926 22:48:22.881992 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.881994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.882009 (+ 15us) server_negotiation.cc:407] Connection header received
0926 22:48:22.882045 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.882046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.882055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.882141 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 22:48:22.882370 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.883146 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.884660 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.885228 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.885297 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.885305 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.885579 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.885634 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:48:22.885685 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.881865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.195:37981 (local address 127.14.184.194:59165)
0926 22:48:22.881928 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.881940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.881983 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.882181 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.882181 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.882183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.882276 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.882278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.883234 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.883235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.884566 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.884569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.885343 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.885345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.885441 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.885447 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.885526 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.885581 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":81}
I20260926 22:48:22.885685 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.881867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:35315 (local address 127.14.184.194:53231)
0926 22:48:22.882043 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.882053 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:22.882103 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:22.882259 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:22.882259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:22.882261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 22:48:22.882314 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.882315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.883234 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.883235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.884566 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:22.884569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.885343 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.885345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:22.885441 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.885448 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 22:48:22.885521 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 22:48:22.885574 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":68,"thread_start_us":99,"threads_started":1}
I20260926 22:48:22.886057 2697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5c42b5a63f4fc68ae1eb48381bcb36" candidate_uuid: "052554ad43344a74be5118f625d84433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96b081572364dfbbc5d1671b2be0b86" is_pre_election: true
I20260926 22:48:22.886107 2435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5c42b5a63f4fc68ae1eb48381bcb36" candidate_uuid: "052554ad43344a74be5118f625d84433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af71f4ba6a947978505822137a49dcc" is_pre_election: true
I20260926 22:48:22.886248 2697 raft_consensus.cc:2454] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052554ad43344a74be5118f625d84433 in term 0.
I20260926 22:48:22.886279 2435 raft_consensus.cc:2454] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052554ad43344a74be5118f625d84433 in term 0.
I20260926 22:48:22.886529 2501 leader_election.cc:302] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052554ad43344a74be5118f625d84433, 8af71f4ba6a947978505822137a49dcc; no voters:
I20260926 22:48:22.886646 2765 raft_consensus.cc:2781] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 22:48:22.886757 2765 raft_consensus.cc:482] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 22:48:22.886778 2765 raft_consensus.cc:3037] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.887504 2765 raft_consensus.cc:504] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.887641 2765 leader_election.cc:288] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [CANDIDATE]: Term 1 election: Requested vote from peers 8af71f4ba6a947978505822137a49dcc (127.14.184.193:35315), e96b081572364dfbbc5d1671b2be0b86 (127.14.184.195:37981)
I20260926 22:48:22.887830 2435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5c42b5a63f4fc68ae1eb48381bcb36" candidate_uuid: "052554ad43344a74be5118f625d84433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af71f4ba6a947978505822137a49dcc"
I20260926 22:48:22.887881 2697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5c42b5a63f4fc68ae1eb48381bcb36" candidate_uuid: "052554ad43344a74be5118f625d84433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96b081572364dfbbc5d1671b2be0b86"
I20260926 22:48:22.887916 2435 raft_consensus.cc:3037] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.887995 2697 raft_consensus.cc:3037] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:22.888669 2435 raft_consensus.cc:2454] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052554ad43344a74be5118f625d84433 in term 1.
I20260926 22:48:22.888708 2697 raft_consensus.cc:2454] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052554ad43344a74be5118f625d84433 in term 1.
I20260926 22:48:22.888909 2501 leader_election.cc:302] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052554ad43344a74be5118f625d84433, 8af71f4ba6a947978505822137a49dcc; no voters:
I20260926 22:48:22.889014 2765 raft_consensus.cc:2781] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:22.889148 2765 raft_consensus.cc:686] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 1 LEADER]: Becoming Leader. State: Replica: 052554ad43344a74be5118f625d84433, State: Running, Role: LEADER
I20260926 22:48:22.889256 2765 consensus_queue.cc:237] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } }
I20260926 22:48:22.890080 2293 catalog_manager.cc:5495] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 reported cstate change: term changed from 0 to 1, leader changed from <none> to 052554ad43344a74be5118f625d84433 (127.14.184.194). New cstate: current_term: 1 leader_uuid: "052554ad43344a74be5118f625d84433" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "052554ad43344a74be5118f625d84433" member_type: VOTER last_known_addr { host: "127.14.184.194" port: 39925 } health_report { overall_health: HEALTHY } } }
I20260926 22:48:22.898514 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.896171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.14.184.194:39925)
0926 22:48:22.896214 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:22.896217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:22.896226 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:22.896301 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:22.896302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:22.896325 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:22.896401 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:22.896527 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.896983 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.897746 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:22.897937 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:22.898012 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:22.898187 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:22.898284 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:22.898377 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:22.898396 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 22:48:22.898413 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:48:22.900127 2697 raft_consensus.cc:1261] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 1 FOLLOWER]: Refusing update from remote peer 052554ad43344a74be5118f625d84433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:22.900127 2435 raft_consensus.cc:1261] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 8af71f4ba6a947978505822137a49dcc [term 1 FOLLOWER]: Refusing update from remote peer 052554ad43344a74be5118f625d84433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:22.900435 2767 consensus_queue.cc:1035] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e96b081572364dfbbc5d1671b2be0b86" member_type: VOTER last_known_addr { host: "127.14.184.195" port: 37981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:22.900563 2765 consensus_queue.cc:1035] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af71f4ba6a947978505822137a49dcc" member_type: VOTER last_known_addr { host: "127.14.184.193" port: 35315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:22.903602 2771 mvcc.cc:205] Tried to move back new op lower bound from 7333736050276605952 to 7333736050235088896. Current Snapshot: MvccSnapshot[applied={T|T < 7333736050276605952}]
I20260926 22:48:22.905180 2292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:22.905274 2292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 22:48:22.905980 2292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b374722ffb1941a4b97a2785e22bb826: Sending DeleteTablet for 3 replicas of tablet 2b5c42b5a63f4fc68ae1eb48381bcb36
I20260926 22:48:22.906296 2415 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:22 UTC) from {username='slave'} at 127.0.0.1:46358
I20260926 22:48:22.906397 2677 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:22 UTC) from {username='slave'} at 127.0.0.1:37646
I20260926 22:48:22.906419 2546 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 22:48:22 UTC) from {username='slave'} at 127.0.0.1:46856
I20260926 22:48:22.906602 2782 tablet_replica.cc:330] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: stopping tablet replica
I20260926 22:48:22.906724 2782 raft_consensus.cc:2229] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 22:48:22.906786 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2353
I20260926 22:48:22.906754 2781 tablet_replica.cc:330] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: stopping tablet replica
I20260926 22:48:22.906836 2782 raft_consensus.cc:2258] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:22.906961 2781 raft_consensus.cc:2229] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 1 LEADER]: Raft consensus shutting down.
I20260926 22:48:22.907083 2781 raft_consensus.cc:2258] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 22:48:22.908057 2782 ts_tablet_manager.cc:1905] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:22.909554 2782 ts_tablet_manager.cc:1918] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:22.909636 2782 log.cc:1196] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-2/wal/wals/2b5c42b5a63f4fc68ae1eb48381bcb36
I20260926 22:48:22.909893 2782 ts_tablet_manager.cc:1939] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P e96b081572364dfbbc5d1671b2be0b86: Deleting consensus metadata
I20260926 22:48:22.910290 2280 catalog_manager.cc:4841] TS e96b081572364dfbbc5d1671b2be0b86 (127.14.184.195:37981): tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 (table test-table [id=bcc12db312194e32b9c19d1acb163c0b]) successfully deleted
I20260926 22:48:22.910425 2781 ts_tablet_manager.cc:1905] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 22:48:22.911686 2781 ts_tablet_manager.cc:1918] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 22:48:22.911782 2781 log.cc:1196] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Deleting WAL directory at /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790462668484683-15075-0/minicluster-data/ts-1/wal/wals/2b5c42b5a63f4fc68ae1eb48381bcb36
I20260926 22:48:22.911999 2781 ts_tablet_manager.cc:1939] T 2b5c42b5a63f4fc68ae1eb48381bcb36 P 052554ad43344a74be5118f625d84433: Deleting consensus metadata
I20260926 22:48:22.912413 2278 catalog_manager.cc:4841] TS 052554ad43344a74be5118f625d84433 (127.14.184.194:39925): tablet 2b5c42b5a63f4fc68ae1eb48381bcb36 (table test-table [id=bcc12db312194e32b9c19d1acb163c0b]) successfully deleted
W20260926 22:48:22.914108 2279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.184.193:35315 (error 108)
I20260926 22:48:22.914570 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:22.914442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.193:35315 (local address 127.0.0.1:46372)
0926 22:48:22.914495 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:22.914534 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.193:35315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:48:22.914654 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2484
W20260926 22:48:22.914736 2279 catalog_manager.cc:4584] TS 8af71f4ba6a947978505822137a49dcc (127.14.184.193:35315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b5c42b5a63f4fc68ae1eb48381bcb36: Network error: Client connection negotiation failed: client connection to 127.14.184.193:35315: connect: Connection refused (error 111)
I20260926 22:48:22.919479 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2615
I20260926 22:48:22.924244 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2262
2026-09-26T22:48:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (697 ms)
[----------] 6 tests from EncryptionPolicyTest (8197 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:22 dist-test-slave-3986 krb5kdc[2786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:22 dist-test-slave-3986 krb5kdc[2786](info): set up 2 sockets
Sep 26 22:48:22 dist-test-slave-3986 krb5kdc[2786](info): commencing operation
krb5kdc: starting...
W20260926 22:48:23.958941 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:24 dist-test-slave-3986 krb5kdc[2786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:24.072978 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:24.152616 2799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:24.152827 2799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:24.152848 2799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:24.154008 2799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:24.154052 2799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:24.154065 2799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:24.154079 2799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:24.154134 2799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:24.155583 2799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:24.155838 2799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:24.156070 2799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 22:48:24.160895 2799 server_base.cc:1034] running on GCE node
W20260926 22:48:24.160844 2807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:24.160859 2805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:24.160818 2804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:24.161275 2799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:24.161573 2799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:24.162734 2799 hybrid_clock.cc:648] HybridClock initialized: now 1790462904162720 us; error 31 us; skew 500 ppm
Sep 26 22:48:24 dist-test-slave-3986 krb5kdc[2786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462904, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:24.165220 2799 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:24.165676 2799 webserver.cc:460] Webserver started at http://127.0.0.1:43003/ using document root <none> and password file <none>
I20260926 22:48:24.165918 2799 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:24.165985 2799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:24.166136 2799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:24.167246 2799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "ede2fd6eccbd40ba8ce917f549772b95"
format_stamp: "Formatted at 2026-09-26 22:48:24 on dist-test-slave-3986"
server_key: "90d95a59c1eacd8c88ef1350e61f70ac"
server_key_iv: "d28e158a93905a5f0973c4ae631d3472"
server_key_version: "encryptionkey@0"
I20260926 22:48:24.167591 2799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "ede2fd6eccbd40ba8ce917f549772b95"
format_stamp: "Formatted at 2026-09-26 22:48:24 on dist-test-slave-3986"
server_key: "90d95a59c1eacd8c88ef1350e61f70ac"
server_key_iv: "d28e158a93905a5f0973c4ae631d3472"
server_key_version: "encryptionkey@0"
I20260926 22:48:24.168886 2799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:24.169615 2814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:24.169818 2799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:24.169920 2799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "ede2fd6eccbd40ba8ce917f549772b95"
format_stamp: "Formatted at 2026-09-26 22:48:24 on dist-test-slave-3986"
server_key: "90d95a59c1eacd8c88ef1350e61f70ac"
server_key_iv: "d28e158a93905a5f0973c4ae631d3472"
server_key_version: "encryptionkey@0"
I20260926 22:48:24.169997 2799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:24.178838 2799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:24.180537 2799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:24.180696 2799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:24.184602 2799 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38109
I20260926 22:48:24.184631 2866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38109 every 8 connection(s)
I20260926 22:48:24.184967 2799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:24.185773 2867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:24.187348 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2799
I20260926 22:48:24.187472 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:24.187659 15075 external_mini_cluster.cc:1428] Setting key baf37073ebc0e7a6a2c5397acc355a86
I20260926 22:48:24.188244 2867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95: Bootstrap starting.
I20260926 22:48:24.188798 2867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:24.189138 2867 log.cc:824] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:24.189881 2867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95: No bootstrap required, opened a new log
I20260926 22:48:24.192003 2867 raft_consensus.cc:348] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede2fd6eccbd40ba8ce917f549772b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38109 } }
I20260926 22:48:24.192127 2867 raft_consensus.cc:374] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:24.192148 2867 raft_consensus.cc:729] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede2fd6eccbd40ba8ce917f549772b95, State: Initialized, Role: FOLLOWER
I20260926 22:48:24.192279 2867 consensus_queue.cc:260] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede2fd6eccbd40ba8ce917f549772b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38109 } }
I20260926 22:48:24.192353 2867 raft_consensus.cc:388] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:24.192374 2867 raft_consensus.cc:482] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:24.192440 2867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:24.192929 2867 raft_consensus.cc:504] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede2fd6eccbd40ba8ce917f549772b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38109 } }
I20260926 22:48:24.193044 2867 leader_election.cc:302] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ede2fd6eccbd40ba8ce917f549772b95; no voters:
I20260926 22:48:24.193243 2867 leader_election.cc:288] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:24.193255 2872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:24.193382 2872 raft_consensus.cc:686] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [term 1 LEADER]: Becoming Leader. State: Replica: ede2fd6eccbd40ba8ce917f549772b95, State: Running, Role: LEADER
I20260926 22:48:24.193461 2872 consensus_queue.cc:237] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede2fd6eccbd40ba8ce917f549772b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38109 } }
I20260926 22:48:24.193509 2867 sys_catalog.cc:564] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:24.193769 2874 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ede2fd6eccbd40ba8ce917f549772b95. Latest consensus state: current_term: 1 leader_uuid: "ede2fd6eccbd40ba8ce917f549772b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede2fd6eccbd40ba8ce917f549772b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38109 } } }
I20260926 22:48:24.194015 2874 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:24.193930 2873 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ede2fd6eccbd40ba8ce917f549772b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede2fd6eccbd40ba8ce917f549772b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38109 } } }
I20260926 22:48:24.194288 2873 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:24.194442 2881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 22:48:24 dist-test-slave-3986 krb5kdc[2786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 22:48:24.195071 2881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:24.196996 2881 catalog_manager.cc:1347] Generated new cluster ID: 78f9a59a9a7b4b109f6d6a81423978d6
I20260926 22:48:24.197045 2881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:24.199124 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:24.188228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39525 (local address 127.0.0.1:38109)
0926 22:48:24.188402 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:24.188409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:24.188432 (+ 23us) server_negotiation.cc:407] Connection header received
0926 22:48:24.188830 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:24.188836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:24.188879 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:24.189054 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:24.189831 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:24.190798 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:24.191905 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:24.192407 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:24.192519 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:24.195302 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:24.195305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:24.195305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:24.195316 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:24.197284 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:24.197679 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:24.197681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:24.197682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:24.197784 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:24.197979 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:24.197979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:24.197979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:24.198230 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:24.198358 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:24.198835 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 22:48:24.198923 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260926 22:48:24.200309 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:24.207018 2881 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:24.207723 2881 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:24.209051 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:24.201878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49475 (local address 127.0.0.1:38109)
0926 22:48:24.201908 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:24.201909 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:24.201916 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:48:24.201988 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:24.201989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:24.201997 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:24.202033 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:24.202641 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:24.203357 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:24.204339 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:24.204649 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:24.204709 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:24.206033 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:24.206035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:24.206035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:24.206041 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:24.207589 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:24.208132 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:24.208133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:24.208133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:24.208179 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:24.208380 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:24.208380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:24.208381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:24.208480 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:24.208559 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:24.208918 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 22:48:24.208973 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 22:48:24.210645 2881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ede2fd6eccbd40ba8ce917f549772b95: Generated new TSK 0
I20260926 22:48:24.210850 2881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:24.214723 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2799
W20260926 22:48:24.218497 2784 connection.cc:535] client connection to 127.0.0.1:38109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38109): Connection reset by peer (error 104)
2026-09-26T22:48:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:24 dist-test-slave-3986 krb5kdc[2898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:24 dist-test-slave-3986 krb5kdc[2898](info): set up 2 sockets
Sep 26 22:48:24 dist-test-slave-3986 krb5kdc[2898](info): commencing operation
krb5kdc: starting...
W20260926 22:48:25.247006 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:25 dist-test-slave-3986 krb5kdc[2898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:25.361714 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:25.439455 2911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:25.439651 2911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:25.439672 2911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:25.440842 2911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:25.440886 2911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:25.440900 2911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:25.440913 2911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:25.440925 2911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:25.442358 2911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:25.442597 2911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:25.442834 2911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:25.447453 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:25.447448 2919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:25.447429 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:25.447697 2911 server_base.cc:1034] running on GCE node
I20260926 22:48:25.447917 2911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:25.448143 2911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:25.449342 2911 hybrid_clock.cc:648] HybridClock initialized: now 1790462905449328 us; error 24 us; skew 500 ppm
Sep 26 22:48:25 dist-test-slave-3986 krb5kdc[2898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462905, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:25.451722 2911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:25.452196 2911 webserver.cc:460] Webserver started at http://127.0.0.1:34097/ using document root <none> and password file <none>
I20260926 22:48:25.452422 2911 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:25.452488 2911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:25.452633 2911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:25.453609 2911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "959da7569fcc4a8aa719b5dbab944a5e"
format_stamp: "Formatted at 2026-09-26 22:48:25 on dist-test-slave-3986"
server_key: "794ebd39481ac07a004811c8fb847038"
server_key_iv: "3819b96079b7353b01d6a4b171eb707c"
server_key_version: "encryptionkey@0"
I20260926 22:48:25.453958 2911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "959da7569fcc4a8aa719b5dbab944a5e"
format_stamp: "Formatted at 2026-09-26 22:48:25 on dist-test-slave-3986"
server_key: "794ebd39481ac07a004811c8fb847038"
server_key_iv: "3819b96079b7353b01d6a4b171eb707c"
server_key_version: "encryptionkey@0"
I20260926 22:48:25.455331 2911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:25.456024 2926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:25.456184 2911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:25.456288 2911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "959da7569fcc4a8aa719b5dbab944a5e"
format_stamp: "Formatted at 2026-09-26 22:48:25 on dist-test-slave-3986"
server_key: "794ebd39481ac07a004811c8fb847038"
server_key_iv: "3819b96079b7353b01d6a4b171eb707c"
server_key_version: "encryptionkey@0"
I20260926 22:48:25.456363 2911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:25.472333 2911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:25.474078 2911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:25.474236 2911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:25.478325 2911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33447
I20260926 22:48:25.478374 2978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33447 every 8 connection(s)
I20260926 22:48:25.478816 2911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:25.479633 2979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:25.481791 2979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e: Bootstrap starting.
I20260926 22:48:25.482306 2979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:25.482529 2979 log.cc:824] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:25.483177 2979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e: No bootstrap required, opened a new log
I20260926 22:48:25.484659 2979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959da7569fcc4a8aa719b5dbab944a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260926 22:48:25.484791 2979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:25.484814 2979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959da7569fcc4a8aa719b5dbab944a5e, State: Initialized, Role: FOLLOWER
I20260926 22:48:25.484911 2979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959da7569fcc4a8aa719b5dbab944a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260926 22:48:25.484963 2979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:25.484983 2979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:25.485018 2979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:25.485483 2979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959da7569fcc4a8aa719b5dbab944a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260926 22:48:25.485563 2979 leader_election.cc:302] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 959da7569fcc4a8aa719b5dbab944a5e; no voters:
I20260926 22:48:25.485700 2979 leader_election.cc:288] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:25.485742 2982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:25.485891 2982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [term 1 LEADER]: Becoming Leader. State: Replica: 959da7569fcc4a8aa719b5dbab944a5e, State: Running, Role: LEADER
I20260926 22:48:25.485975 2979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:25.486018 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 2911
I20260926 22:48:25.486017 2982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959da7569fcc4a8aa719b5dbab944a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } }
I20260926 22:48:25.486172 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:25.486315 2982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959da7569fcc4a8aa719b5dbab944a5e. Latest consensus state: current_term: 1 leader_uuid: "959da7569fcc4a8aa719b5dbab944a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959da7569fcc4a8aa719b5dbab944a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } } }
I20260926 22:48:25.486368 15075 external_mini_cluster.cc:1428] Setting key 536497136230ea502a623be2d1ae5a12
I20260926 22:48:25.486387 2982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:25.486462 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "959da7569fcc4a8aa719b5dbab944a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959da7569fcc4a8aa719b5dbab944a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33447 } } }
I20260926 22:48:25.486513 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:25.486788 2987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:25.487265 2987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:25.489615 2987 catalog_manager.cc:1347] Generated new cluster ID: 4d1b07d953564e4fb8d83e438fd8a562
I20260926 22:48:25.489681 2987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:25.492400 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:25.487239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36625 (local address 127.0.0.1:33447)
0926 22:48:25.487455 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:25.487464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:25.487500 (+ 36us) server_negotiation.cc:407] Connection header received
0926 22:48:25.487619 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:25.487722 (+ 103us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:25.487777 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:25.487926 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:25.488127 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:25.489190 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:25.490322 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:25.490940 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:25.491043 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:25.491752 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:25.491756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:25.491756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:25.491948 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:25.491982 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:25.491995 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:25.492070 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:25.492184 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 22:48:25.492207 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260926 22:48:25.493337 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:25.506239 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:25.494970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40325 (local address 127.0.0.1:33447)
0926 22:48:25.495004 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:25.495006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:25.502844 (+ 7838us) server_negotiation.cc:407] Connection header received
0926 22:48:25.502875 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:25.502876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:25.502890 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:25.502961 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:25.503106 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:25.503876 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:25.504847 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:25.505210 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:25.505281 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:25.505987 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:25.505989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:25.505989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:25.505993 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:25.506009 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:25.506014 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:25.506077 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:25.506146 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 22:48:25.506160 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:25.531656 2987 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:25.532197 2987 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:25.536190 2987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 959da7569fcc4a8aa719b5dbab944a5e: Generated new TSK 0
I20260926 22:48:25.536396 2987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:25.545836 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 2911
W20260926 22:48:25.550345 2896 connection.cc:535] client connection to 127.0.0.1:33447 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33447): Connection reset by peer (error 104)
2026-09-26T22:48:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:25 dist-test-slave-3986 krb5kdc[3010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:25 dist-test-slave-3986 krb5kdc[3010](info): set up 2 sockets
Sep 26 22:48:25 dist-test-slave-3986 krb5kdc[3010](info): commencing operation
krb5kdc: starting...
W20260926 22:48:26.567000 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:26 dist-test-slave-3986 krb5kdc[3010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:26.677090 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:26.756120 3023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:26.756309 3023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:26.756331 3023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:26.757403 3023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:26.757462 3023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:26.757476 3023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:26.757489 3023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:26.757525 3023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:26.758988 3023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37571
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:26.759217 3023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:26.759425 3023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:26.764127 3029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:26.764138 3031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:26.764138 3028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:26.764213 3023 server_base.cc:1034] running on GCE node
I20260926 22:48:26.764568 3023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:26.764842 3023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:26.765982 3023 hybrid_clock.cc:648] HybridClock initialized: now 1790462906765973 us; error 27 us; skew 500 ppm
Sep 26 22:48:26 dist-test-slave-3986 krb5kdc[3010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462906, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:26.771831 3023 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:26.772285 3023 webserver.cc:460] Webserver started at http://127.0.0.1:38121/ using document root <none> and password file <none>
I20260926 22:48:26.772516 3023 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:26.772558 3023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:26.772749 3023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:26.773763 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "c659189029e84c7cb8f21050fe8c496d"
format_stamp: "Formatted at 2026-09-26 22:48:26 on dist-test-slave-3986"
server_key: "805984d532f07659482f4d59799cd6c2"
server_key_iv: "26d1c251b79584a8defe942f268a7c08"
server_key_version: "encryptionkey@0"
I20260926 22:48:26.774111 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "c659189029e84c7cb8f21050fe8c496d"
format_stamp: "Formatted at 2026-09-26 22:48:26 on dist-test-slave-3986"
server_key: "805984d532f07659482f4d59799cd6c2"
server_key_iv: "26d1c251b79584a8defe942f268a7c08"
server_key_version: "encryptionkey@0"
I20260926 22:48:26.775477 3023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:26.776177 3038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:26.776371 3023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:26.776461 3023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "c659189029e84c7cb8f21050fe8c496d"
format_stamp: "Formatted at 2026-09-26 22:48:26 on dist-test-slave-3986"
server_key: "805984d532f07659482f4d59799cd6c2"
server_key_iv: "26d1c251b79584a8defe942f268a7c08"
server_key_version: "encryptionkey@0"
I20260926 22:48:26.776532 3023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:26.779762 3023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:26.781572 3023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:26.781718 3023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:26.785635 3023 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37571
I20260926 22:48:26.785684 3090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37571 every 8 connection(s)
I20260926 22:48:26.785986 3023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:26.786567 3091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:26.788741 3091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d: Bootstrap starting.
I20260926 22:48:26.789265 3091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:26.789510 3091 log.cc:824] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:26.790135 3091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d: No bootstrap required, opened a new log
I20260926 22:48:26.791131 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3023
I20260926 22:48:26.791249 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:26.791443 15075 external_mini_cluster.cc:1428] Setting key aa73aeff18da5c736205677353b6fce8
I20260926 22:48:26.792338 3091 raft_consensus.cc:348] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659189029e84c7cb8f21050fe8c496d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37571 } }
I20260926 22:48:26.792538 3091 raft_consensus.cc:374] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:26.792582 3091 raft_consensus.cc:729] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c659189029e84c7cb8f21050fe8c496d, State: Initialized, Role: FOLLOWER
I20260926 22:48:26.792764 3091 consensus_queue.cc:260] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659189029e84c7cb8f21050fe8c496d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37571 } }
I20260926 22:48:26.792883 3091 raft_consensus.cc:388] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:26.792927 3091 raft_consensus.cc:482] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:26.792984 3091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:26.793628 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:26.791948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54147 (local address 127.0.0.1:37571)
0926 22:48:26.792114 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:26.792121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:26.792145 (+ 24us) server_negotiation.cc:407] Connection header received
0926 22:48:26.792295 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:26.792307 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:26.792329 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:26.792452 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:26.793144 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:26.793146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:26.793146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:26.793150 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:26.793184 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:26.793194 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:26.793241 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:26.793365 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 22:48:26.793396 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260926 22:48:26.794185 3091 raft_consensus.cc:504] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659189029e84c7cb8f21050fe8c496d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37571 } }
I20260926 22:48:26.794297 3091 leader_election.cc:302] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c659189029e84c7cb8f21050fe8c496d; no voters:
I20260926 22:48:26.794452 3091 leader_election.cc:288] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:26.794498 3096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:26.794648 3096 raft_consensus.cc:686] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [term 1 LEADER]: Becoming Leader. State: Replica: c659189029e84c7cb8f21050fe8c496d, State: Running, Role: LEADER
I20260926 22:48:26.794682 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:26.794729 3091 sys_catalog.cc:564] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:26.794818 3096 consensus_queue.cc:237] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659189029e84c7cb8f21050fe8c496d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37571 } }
I20260926 22:48:26.795140 3097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c659189029e84c7cb8f21050fe8c496d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659189029e84c7cb8f21050fe8c496d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37571 } } }
I20260926 22:48:26.795239 3097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:26.795131 3096 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c659189029e84c7cb8f21050fe8c496d. Latest consensus state: current_term: 1 leader_uuid: "c659189029e84c7cb8f21050fe8c496d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c659189029e84c7cb8f21050fe8c496d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37571 } } }
I20260926 22:48:26.795517 3096 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:26.795943 3103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:26.796485 3103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:26.797699 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:26.796262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55777 (local address 127.0.0.1:37571)
0926 22:48:26.796320 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:26.796322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:26.796547 (+ 225us) server_negotiation.cc:407] Connection header received
0926 22:48:26.796749 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:26.796750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:26.796755 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:26.796786 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:26.797477 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:26.797478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:26.797478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:26.797483 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:26.797496 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:26.797500 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:26.797559 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:26.797607 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 22:48:26.797621 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:48:26.798439 3103 catalog_manager.cc:1347] Generated new cluster ID: e4df1f0b77d640a2870314745d5585fd
I20260926 22:48:26.798491 3103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:26.810508 3103 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:26.810681 3103 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:26.820220 3103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c659189029e84c7cb8f21050fe8c496d: Generated new TSK 0
I20260926 22:48:26.820410 3103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:26.828073 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3023
2026-09-26T22:48:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:26 dist-test-slave-3986 krb5kdc[3122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:26 dist-test-slave-3986 krb5kdc[3122](info): set up 2 sockets
Sep 26 22:48:26 dist-test-slave-3986 krb5kdc[3122](info): commencing operation
krb5kdc: starting...
W20260926 22:48:27.862946 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:27 dist-test-slave-3986 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:27.975703 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:28.055074 3135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:28.055260 3135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:28.055284 3135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:28.056396 3135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:28.056445 3135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:28.056458 3135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:28.056494 3135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:28.056523 3135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:28.057960 3135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:28.058188 3135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:28.058418 3135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:28.063256 3140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:28.063267 3143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:28.063267 3141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:28.063330 3135 server_base.cc:1034] running on GCE node
I20260926 22:48:28.063746 3135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:28.064023 3135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:28.065181 3135 hybrid_clock.cc:648] HybridClock initialized: now 1790462908065168 us; error 40 us; skew 500 ppm
Sep 26 22:48:28 dist-test-slave-3986 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462908, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:28.067512 3135 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:28.067951 3135 webserver.cc:460] Webserver started at http://127.0.0.1:41991/ using document root <none> and password file <none>
I20260926 22:48:28.068181 3135 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:28.068264 3135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:28.068418 3135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:28.069407 3135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "3113db4957c441bcb48f21cbcf0ee608"
format_stamp: "Formatted at 2026-09-26 22:48:28 on dist-test-slave-3986"
server_key: "ecf3cfe7a07df2d888f1ff0541006b55"
server_key_iv: "535cf6ac4ddf6d8d5acb625014b5da2e"
server_key_version: "encryptionkey@0"
I20260926 22:48:28.069759 3135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "3113db4957c441bcb48f21cbcf0ee608"
format_stamp: "Formatted at 2026-09-26 22:48:28 on dist-test-slave-3986"
server_key: "ecf3cfe7a07df2d888f1ff0541006b55"
server_key_iv: "535cf6ac4ddf6d8d5acb625014b5da2e"
server_key_version: "encryptionkey@0"
I20260926 22:48:28.071107 3135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:28.071812 3150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:28.072005 3135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:28.072108 3135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "3113db4957c441bcb48f21cbcf0ee608"
format_stamp: "Formatted at 2026-09-26 22:48:28 on dist-test-slave-3986"
server_key: "ecf3cfe7a07df2d888f1ff0541006b55"
server_key_iv: "535cf6ac4ddf6d8d5acb625014b5da2e"
server_key_version: "encryptionkey@0"
I20260926 22:48:28.072180 3135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:28.098439 3135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:28.100347 3135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:28.100512 3135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:28.104430 3135 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43481
I20260926 22:48:28.104467 3202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43481 every 8 connection(s)
I20260926 22:48:28.104851 3135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:28.105690 3203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:28.107875 3203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608: Bootstrap starting.
I20260926 22:48:28.108389 3203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:28.108613 3203 log.cc:824] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:28.109218 3203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608: No bootstrap required, opened a new log
I20260926 22:48:28.110318 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3135
I20260926 22:48:28.110414 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:28.110598 15075 external_mini_cluster.cc:1428] Setting key c6d9e5cd8a57d8f2a2dbd52f6b2a417f
I20260926 22:48:28.111063 3203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3113db4957c441bcb48f21cbcf0ee608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43481 } }
I20260926 22:48:28.111261 3203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:28.111366 3203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3113db4957c441bcb48f21cbcf0ee608, State: Initialized, Role: FOLLOWER
I20260926 22:48:28.111500 3203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3113db4957c441bcb48f21cbcf0ee608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43481 } }
I20260926 22:48:28.111598 3203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:28.111640 3203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:28.111728 3203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:28.112946 3203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3113db4957c441bcb48f21cbcf0ee608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43481 } }
I20260926 22:48:28.113070 3203 leader_election.cc:302] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3113db4957c441bcb48f21cbcf0ee608; no voters:
I20260926 22:48:28.113279 3203 leader_election.cc:288] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:28.113299 3208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:28.113498 3208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [term 1 LEADER]: Becoming Leader. State: Replica: 3113db4957c441bcb48f21cbcf0ee608, State: Running, Role: LEADER
I20260926 22:48:28.113626 3208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3113db4957c441bcb48f21cbcf0ee608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43481 } }
I20260926 22:48:28.113615 3203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:28.114066 3210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3113db4957c441bcb48f21cbcf0ee608. Latest consensus state: current_term: 1 leader_uuid: "3113db4957c441bcb48f21cbcf0ee608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3113db4957c441bcb48f21cbcf0ee608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43481 } } }
I20260926 22:48:28.114287 3210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:28.114225 3209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3113db4957c441bcb48f21cbcf0ee608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3113db4957c441bcb48f21cbcf0ee608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43481 } } }
I20260926 22:48:28.114534 3209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:28.114569 3213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:28.115157 3213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:28.117028 3213 catalog_manager.cc:1347] Generated new cluster ID: 938ab8fe806d4da5a7f77e144d98696c
I20260926 22:48:28.117082 3213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:28 dist-test-slave-3986 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 22:48:28.121570 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:28.111131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38223 (local address 127.0.0.1:43481)
0926 22:48:28.111356 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:28.111365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:28.111394 (+ 29us) server_negotiation.cc:407] Connection header received
0926 22:48:28.111521 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:28.111527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:28.111576 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:28.111711 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:28.112540 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:28.113898 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:28.115092 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:28.115708 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:28.115827 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:28.118384 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:28.118388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:28.118388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:28.118398 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:28.120078 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:28.120457 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:28.120460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:28.120460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:28.120567 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:28.120776 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:28.120777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:28.120777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:28.121001 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:28.121171 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:28.121237 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 22:48:28.121388 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260926 22:48:28.122444 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:28.131595 3213 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:28.132273 3213 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:28.132994 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:28.123848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39175 (local address 127.0.0.1:43481)
0926 22:48:28.123887 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:28.123888 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:28.126852 (+ 2964us) server_negotiation.cc:407] Connection header received
0926 22:48:28.126926 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:28.126927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:28.126948 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:28.126999 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:28.127542 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:28.128027 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:28.128745 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:28.128959 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:28.129035 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:28.130187 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:28.130188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:28.130189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:28.130197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:28.131846 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:28.132195 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:28.132196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:28.132197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:28.132248 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:28.132465 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:28.132466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:28.132466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:28.132577 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:28.132692 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:28.132792 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 22:48:28.132887 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:48:28.139161 3213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3113db4957c441bcb48f21cbcf0ee608: Generated new TSK 0
I20260926 22:48:28.139333 3213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:28.144446 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3135
2026-09-26T22:48:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:28 dist-test-slave-3986 krb5kdc[3234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:28 dist-test-slave-3986 krb5kdc[3234](info): set up 2 sockets
Sep 26 22:48:28 dist-test-slave-3986 krb5kdc[3234](info): commencing operation
krb5kdc: starting...
W20260926 22:48:29.179100 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:29 dist-test-slave-3986 krb5kdc[3234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:29.294557 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:29.374783 3247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:29.374979 3247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:29.375001 3247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:29.376142 3247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:29.376183 3247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:29.376196 3247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:29.376209 3247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:29.376220 3247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:29.377682 3247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:29.377944 3247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:29.378158 3247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:29.382946 3255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:29.382956 3252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:29.383003 3247 server_base.cc:1034] running on GCE node
W20260926 22:48:29.382962 3253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:29.383435 3247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:29.383711 3247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:29.384883 3247 hybrid_clock.cc:648] HybridClock initialized: now 1790462909384868 us; error 28 us; skew 500 ppm
Sep 26 22:48:29 dist-test-slave-3986 krb5kdc[3234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462909, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:29.387320 3247 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:29.387745 3247 webserver.cc:460] Webserver started at http://127.0.0.1:40237/ using document root <none> and password file <none>
I20260926 22:48:29.387974 3247 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:29.388018 3247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:29.388182 3247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:29.389225 3247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "034498b1969a4f0ca1275acb66f9de34"
format_stamp: "Formatted at 2026-09-26 22:48:29 on dist-test-slave-3986"
server_key: "c5b819f63e0688f3abe2fb014e961b10"
server_key_iv: "96ed364eae26b726ab198dba665c546d"
server_key_version: "encryptionkey@0"
I20260926 22:48:29.389609 3247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "034498b1969a4f0ca1275acb66f9de34"
format_stamp: "Formatted at 2026-09-26 22:48:29 on dist-test-slave-3986"
server_key: "c5b819f63e0688f3abe2fb014e961b10"
server_key_iv: "96ed364eae26b726ab198dba665c546d"
server_key_version: "encryptionkey@0"
I20260926 22:48:29.390962 3247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:29.391672 3262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:29.391862 3247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:29.391953 3247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "034498b1969a4f0ca1275acb66f9de34"
format_stamp: "Formatted at 2026-09-26 22:48:29 on dist-test-slave-3986"
server_key: "c5b819f63e0688f3abe2fb014e961b10"
server_key_iv: "96ed364eae26b726ab198dba665c546d"
server_key_version: "encryptionkey@0"
I20260926 22:48:29.392025 3247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:29.400852 3247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:29.402845 3247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:29.403003 3247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:29.406855 3247 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40145
I20260926 22:48:29.406929 3314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40145 every 8 connection(s)
I20260926 22:48:29.407172 3247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:29.407938 3315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:29.409200 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3247
I20260926 22:48:29.409305 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:29.409503 15075 external_mini_cluster.cc:1428] Setting key ef9233dc142ca2d981c8d12b64bc313a
I20260926 22:48:29.410709 3315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34: Bootstrap starting.
I20260926 22:48:29.411415 3315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:29.411724 3315 log.cc:824] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:29.412441 3315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34: No bootstrap required, opened a new log
I20260926 22:48:29.414649 3315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034498b1969a4f0ca1275acb66f9de34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260926 22:48:29.414857 3315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:29.414901 3315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034498b1969a4f0ca1275acb66f9de34, State: Initialized, Role: FOLLOWER
I20260926 22:48:29.415031 3315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034498b1969a4f0ca1275acb66f9de34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260926 22:48:29.415115 3315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:29.415146 3315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:29.415241 3315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:29.415510 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:29.410009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51365 (local address 127.0.0.1:40145)
0926 22:48:29.410214 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:29.410225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:29.410262 (+ 37us) server_negotiation.cc:407] Connection header received
0926 22:48:29.410854 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:29.410862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:29.410921 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:29.411082 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:29.411221 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:29.412601 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:29.413352 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:29.413984 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:29.414118 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:29.414724 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:29.414727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:29.414728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:29.414923 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:29.414962 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:29.414976 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:29.415034 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:29.415112 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 22:48:29.415276 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260926 22:48:29.415963 3315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034498b1969a4f0ca1275acb66f9de34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260926 22:48:29.416079 3315 leader_election.cc:302] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 034498b1969a4f0ca1275acb66f9de34; no voters:
I20260926 22:48:29.416244 3315 leader_election.cc:288] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:29.416285 3320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:29.416513 3315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:29.416566 3320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [term 1 LEADER]: Becoming Leader. State: Replica: 034498b1969a4f0ca1275acb66f9de34, State: Running, Role: LEADER
I20260926 22:48:29.416612 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:29.416688 3320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034498b1969a4f0ca1275acb66f9de34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260926 22:48:29.417013 3321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "034498b1969a4f0ca1275acb66f9de34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034498b1969a4f0ca1275acb66f9de34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } } }
I20260926 22:48:29.417145 3321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:29.417011 3322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 034498b1969a4f0ca1275acb66f9de34. Latest consensus state: current_term: 1 leader_uuid: "034498b1969a4f0ca1275acb66f9de34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034498b1969a4f0ca1275acb66f9de34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } } }
I20260926 22:48:29.417434 3322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:29.417685 3329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:29.418192 3329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:29.420435 3329 catalog_manager.cc:1347] Generated new cluster ID: 7f75b8eba3f1444eb68f7a2b8ef0f883
I20260926 22:48:29.420488 3329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:29.421363 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:29.418131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58347 (local address 127.0.0.1:40145)
0926 22:48:29.418163 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:29.418165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:29.418341 (+ 176us) server_negotiation.cc:407] Connection header received
0926 22:48:29.418432 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:29.418433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:29.418442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:29.418486 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:29.418625 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:29.419349 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:29.420053 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:29.420320 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:29.420384 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:29.421108 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:29.421110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:29.421110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:29.421114 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:29.421127 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:29.421131 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:29.421185 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:29.421243 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 22:48:29.421264 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 22:48:29.434198 3329 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:29.434770 3329 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:29.439834 3329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 034498b1969a4f0ca1275acb66f9de34: Generated new TSK 0
I20260926 22:48:29.440014 3329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:29.445334 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3247
2026-09-26T22:48:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:29 dist-test-slave-3986 krb5kdc[3346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:29 dist-test-slave-3986 krb5kdc[3346](info): set up 2 sockets
Sep 26 22:48:29 dist-test-slave-3986 krb5kdc[3346](info): commencing operation
krb5kdc: starting...
W20260926 22:48:30.474933 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:30 dist-test-slave-3986 krb5kdc[3346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:30.588985 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:30.666829 3359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:30.667022 3359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:30.667045 3359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:30.668192 3359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:30.668233 3359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:30.668247 3359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:30.668287 3359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:30.668318 3359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:30.669725 3359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:30.669960 3359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:30.670174 3359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:30.675073 3365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:30.675078 3367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:30.675109 3359 server_base.cc:1034] running on GCE node
W20260926 22:48:30.675078 3364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:30.675529 3359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:30.675804 3359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:30.676970 3359 hybrid_clock.cc:648] HybridClock initialized: now 1790462910676954 us; error 66 us; skew 500 ppm
Sep 26 22:48:30 dist-test-slave-3986 krb5kdc[3346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:30.679301 3359 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:30.679706 3359 webserver.cc:460] Webserver started at http://127.0.0.1:38001/ using document root <none> and password file <none>
I20260926 22:48:30.679929 3359 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:30.679994 3359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:30.680137 3359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:30.681149 3359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "b1a7cc6563d24ded90d2ed815c5fab8b"
format_stamp: "Formatted at 2026-09-26 22:48:30 on dist-test-slave-3986"
server_key: "562395ac0004ad766c830abc21e6ac55"
server_key_iv: "7ceb28f27cc5681dd92153ed8c277e80"
server_key_version: "encryptionkey@0"
I20260926 22:48:30.681500 3359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "b1a7cc6563d24ded90d2ed815c5fab8b"
format_stamp: "Formatted at 2026-09-26 22:48:30 on dist-test-slave-3986"
server_key: "562395ac0004ad766c830abc21e6ac55"
server_key_iv: "7ceb28f27cc5681dd92153ed8c277e80"
server_key_version: "encryptionkey@0"
I20260926 22:48:30.682803 3359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:30.683558 3374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:30.683727 3359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:30.683831 3359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "b1a7cc6563d24ded90d2ed815c5fab8b"
format_stamp: "Formatted at 2026-09-26 22:48:30 on dist-test-slave-3986"
server_key: "562395ac0004ad766c830abc21e6ac55"
server_key_iv: "7ceb28f27cc5681dd92153ed8c277e80"
server_key_version: "encryptionkey@0"
I20260926 22:48:30.683910 3359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:30.687099 3359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:30.688792 3359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:30.688947 3359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:30.692675 3359 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44821
I20260926 22:48:30.692703 3426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44821 every 8 connection(s)
I20260926 22:48:30.693017 3359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:30.693257 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3359
I20260926 22:48:30.693369 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:30.693537 15075 external_mini_cluster.cc:1428] Setting key 7c09bf862a2e875c46a920960bcc867f
I20260926 22:48:30.694119 3427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:30.695842 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:30.693967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35875 (local address 127.0.0.1:44821)
0926 22:48:30.694134 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:30.694143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:30.694173 (+ 30us) server_negotiation.cc:407] Connection header received
0926 22:48:30.694607 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:30.694614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:30.694649 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:30.694838 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:30.695447 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:30.695450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:30.695450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:30.695454 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:30.695498 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:30.695508 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:30.695581 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:30.695648 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 22:48:30.695686 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260926 22:48:30.696717 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:30.697083 3427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b: Bootstrap starting.
I20260926 22:48:30.697710 3427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:30.698035 3427 log.cc:824] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:30.698804 3427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b: No bootstrap required, opened a new log
I20260926 22:48:30.699085 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:30.698056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41231 (local address 127.0.0.1:44821)
0926 22:48:30.698090 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:30.698092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:30.698159 (+ 67us) server_negotiation.cc:407] Connection header received
0926 22:48:30.698260 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:30.698261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:30.698266 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:30.698324 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:30.698866 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:30.698867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:30.698867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:30.698871 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:30.698885 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:30.698889 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:30.698950 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:30.699009 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 22:48:30.699020 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:48:30.700814 3427 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44821 } }
I20260926 22:48:30.700937 3427 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:30.700959 3427 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a7cc6563d24ded90d2ed815c5fab8b, State: Initialized, Role: FOLLOWER
I20260926 22:48:30.701098 3427 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44821 } }
I20260926 22:48:30.701174 3427 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:30.701218 3427 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:30.701267 3427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:30.701797 3427 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44821 } }
I20260926 22:48:30.701905 3427 leader_election.cc:302] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1a7cc6563d24ded90d2ed815c5fab8b; no voters:
I20260926 22:48:30.702061 3427 leader_election.cc:288] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:30.702106 3437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:30.702260 3437 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [term 1 LEADER]: Becoming Leader. State: Replica: b1a7cc6563d24ded90d2ed815c5fab8b, State: Running, Role: LEADER
I20260926 22:48:30.702291 3427 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:30.702410 3437 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44821 } }
I20260926 22:48:30.702790 3438 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44821 } } }
I20260926 22:48:30.702907 3438 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:30.703087 3439 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a7cc6563d24ded90d2ed815c5fab8b. Latest consensus state: current_term: 1 leader_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a7cc6563d24ded90d2ed815c5fab8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44821 } } }
I20260926 22:48:30.703183 3439 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b [sys.catalog]: This master's current role is: LEADER
W20260926 22:48:30.704092 3453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:30.704149 3453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:30.704245 3446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:30.704444 3446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:30.705873 3446 catalog_manager.cc:1347] Generated new cluster ID: ebeb74d7f1c24abca0e9cf5bdd76b615
I20260926 22:48:30.705932 3446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:30.720713 3446 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:30.720935 3446 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:30.729341 3446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1a7cc6563d24ded90d2ed815c5fab8b: Generated new TSK 0
I20260926 22:48:30.729540 3446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:30.732775 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3359
2026-09-26T22:48:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:30 dist-test-slave-3986 krb5kdc[3458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:30 dist-test-slave-3986 krb5kdc[3458](info): set up 2 sockets
Sep 26 22:48:30 dist-test-slave-3986 krb5kdc[3458](info): commencing operation
krb5kdc: starting...
W20260926 22:48:31.751055 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:31 dist-test-slave-3986 krb5kdc[3458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:31.865146 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:31.943538 3471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:31.943742 3471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:31.943764 3471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:31.944890 3471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:31.944933 3471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:31.944947 3471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:31.944959 3471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:31.945009 3471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:31.946458 3471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:31.946715 3471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:31.946933 3471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:31.951535 3476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:31.951560 3479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:31.951565 3477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:31.951723 3471 server_base.cc:1034] running on GCE node
I20260926 22:48:31.952049 3471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:31.952304 3471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:31.953452 3471 hybrid_clock.cc:648] HybridClock initialized: now 1790462911953439 us; error 27 us; skew 500 ppm
Sep 26 22:48:31 dist-test-slave-3986 krb5kdc[3458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:31.960677 3471 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:31.961046 3471 webserver.cc:460] Webserver started at http://127.0.0.1:39075/ using document root <none> and password file <none>
I20260926 22:48:31.961225 3471 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:31.961263 3471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:31.961357 3471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:31.962354 3471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba"
format_stamp: "Formatted at 2026-09-26 22:48:31 on dist-test-slave-3986"
server_key: "8e3812b53ab36ce8034e525b3830e933"
server_key_iv: "e73637a9dff67423d71fb91421a0bf60"
server_key_version: "encryptionkey@0"
I20260926 22:48:31.962651 3471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba"
format_stamp: "Formatted at 2026-09-26 22:48:31 on dist-test-slave-3986"
server_key: "8e3812b53ab36ce8034e525b3830e933"
server_key_iv: "e73637a9dff67423d71fb91421a0bf60"
server_key_version: "encryptionkey@0"
I20260926 22:48:31.963958 3471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 22:48:31.964623 3486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:31.964805 3471 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:31.964890 3471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba"
format_stamp: "Formatted at 2026-09-26 22:48:31 on dist-test-slave-3986"
server_key: "8e3812b53ab36ce8034e525b3830e933"
server_key_iv: "e73637a9dff67423d71fb91421a0bf60"
server_key_version: "encryptionkey@0"
I20260926 22:48:31.964946 3471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:31.995934 3471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:31.997668 3471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:31.997771 3471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:32.001466 3471 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43365
I20260926 22:48:32.001521 3538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43365 every 8 connection(s)
I20260926 22:48:32.001840 3471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:32.002738 3539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:32.004902 3539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba: Bootstrap starting.
I20260926 22:48:32.005473 3539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:32.005775 3539 log.cc:824] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:32.006443 3539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba: No bootstrap required, opened a new log
I20260926 22:48:32.007978 3539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43365 } }
I20260926 22:48:32.008137 3539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:32.008188 3539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6c3ab2fcef47dab0b7cb5f4a2289ba, State: Initialized, Role: FOLLOWER
I20260926 22:48:32.008342 3539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43365 } }
I20260926 22:48:32.008424 3539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:32.008466 3539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:32.008517 3539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:32.009039 3539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43365 } }
I20260926 22:48:32.009151 3539 leader_election.cc:302] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a6c3ab2fcef47dab0b7cb5f4a2289ba; no voters:
I20260926 22:48:32.009341 3539 leader_election.cc:288] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:32.009359 3542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:32.009439 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3471
I20260926 22:48:32.009509 3542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [term 1 LEADER]: Becoming Leader. State: Replica: 8a6c3ab2fcef47dab0b7cb5f4a2289ba, State: Running, Role: LEADER
I20260926 22:48:32.009549 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:32.009581 3542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43365 } }
I20260926 22:48:32.009609 3539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:32.009756 15075 external_mini_cluster.cc:1428] Setting key a412389f109946c229647871121ac319
I20260926 22:48:32.010170 3543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43365 } } }
I20260926 22:48:32.010331 3543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:32.010311 3544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a6c3ab2fcef47dab0b7cb5f4a2289ba. Latest consensus state: current_term: 1 leader_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6c3ab2fcef47dab0b7cb5f4a2289ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43365 } } }
I20260926 22:48:32.010545 3544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:32.011008 3553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:32.011555 3553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:32.013044 3553 catalog_manager.cc:1347] Generated new cluster ID: 4feeff5c909d453ebbb4f741e4830c44
I20260926 22:48:32.013092 3553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:32 dist-test-slave-3986 krb5kdc[3458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 22:48:32.022048 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:32.010410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:36459 (local address 127.0.0.1:43365)
0926 22:48:32.010814 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:32.010825 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:32.010864 (+ 39us) server_negotiation.cc:407] Connection header received
0926 22:48:32.010996 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:32.011004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:32.011129 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:32.011306 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:32.012017 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:32.013356 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:32.014386 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:32.015035 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:32.015150 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:32.017854 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:32.017857 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:32.017858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:32.017869 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:32.020312 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:32.020770 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:32.020773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:32.020773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:32.020871 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:32.021131 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:32.021132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:32.021132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:32.021429 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:32.021580 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:32.021715 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 22:48:32.021807 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":99,"threads_started":1}
I20260926 22:48:32.022800 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:32.030975 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:32.024198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:47547 (local address 127.0.0.1:43365)
0926 22:48:32.024231 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:32.024232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:32.024341 (+ 109us) server_negotiation.cc:407] Connection header received
0926 22:48:32.024420 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:32.024420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:32.024438 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:32.024462 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:32.025058 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:32.025802 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:32.026776 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:32.027085 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:32.027121 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:32.028405 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:32.028406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:32.028407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:32.028413 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:32.029787 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:32.030178 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:32.030178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:32.030179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:32.030223 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:32.030441 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:32.030442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:32.030442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:32.030542 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:32.030613 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:32.030836 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 22:48:32.030895 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:48:32.035799 3553 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:32.036370 3553 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:32.042492 3553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a6c3ab2fcef47dab0b7cb5f4a2289ba: Generated new TSK 0
I20260926 22:48:32.042657 3553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:32.049537 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3471
W20260926 22:48:32.053846 3456 connection.cc:535] client connection to 127.0.0.1:43365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43365): Connection reset by peer (error 104)
2026-09-26T22:48:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:32 dist-test-slave-3986 krb5kdc[3570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:32 dist-test-slave-3986 krb5kdc[3570](info): set up 2 sockets
Sep 26 22:48:32 dist-test-slave-3986 krb5kdc[3570](info): commencing operation
krb5kdc: starting...
W20260926 22:48:33.083110 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:33 dist-test-slave-3986 krb5kdc[3570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:33.195245 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:33.272855 3583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:33.273080 3583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:33.273120 3583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:33.274230 3583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:33.274299 3583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:33.274333 3583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:33.274365 3583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:33.274389 3583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:33.275851 3583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:33.276113 3583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:33.276335 3583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:33.280984 3588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:33.280985 3589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:33.281010 3591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:33.281697 3583 server_base.cc:1034] running on GCE node
I20260926 22:48:33.281919 3583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:33.282181 3583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:33.283332 3583 hybrid_clock.cc:648] HybridClock initialized: now 1790462913283318 us; error 30 us; skew 500 ppm
Sep 26 22:48:33 dist-test-slave-3986 krb5kdc[3570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462913, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:33.285620 3583 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:33.286080 3583 webserver.cc:460] Webserver started at http://127.0.0.1:35301/ using document root <none> and password file <none>
I20260926 22:48:33.286311 3583 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:33.286376 3583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:33.286535 3583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:33.287591 3583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "14cc4602a43649e69cb7fd93dd9cb66a"
format_stamp: "Formatted at 2026-09-26 22:48:33 on dist-test-slave-3986"
server_key: "1ea268deb3af24c44d49a2282675e3c6"
server_key_iv: "b29b8931ea931adec89d00abd952defd"
server_key_version: "encryptionkey@0"
I20260926 22:48:33.287950 3583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "14cc4602a43649e69cb7fd93dd9cb66a"
format_stamp: "Formatted at 2026-09-26 22:48:33 on dist-test-slave-3986"
server_key: "1ea268deb3af24c44d49a2282675e3c6"
server_key_iv: "b29b8931ea931adec89d00abd952defd"
server_key_version: "encryptionkey@0"
I20260926 22:48:33.289250 3583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:33.289991 3598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:33.290167 3583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:33.290266 3583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "14cc4602a43649e69cb7fd93dd9cb66a"
format_stamp: "Formatted at 2026-09-26 22:48:33 on dist-test-slave-3986"
server_key: "1ea268deb3af24c44d49a2282675e3c6"
server_key_iv: "b29b8931ea931adec89d00abd952defd"
server_key_version: "encryptionkey@0"
I20260926 22:48:33.290364 3583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:33.310510 3583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:33.312250 3583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:33.312417 3583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:33.316308 3583 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45241
I20260926 22:48:33.316344 3650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45241 every 8 connection(s)
I20260926 22:48:33.316675 3583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:33.317456 3651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:33.319444 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3583
I20260926 22:48:33.319576 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:33.319597 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a: Bootstrap starting.
I20260926 22:48:33.319779 15075 external_mini_cluster.cc:1428] Setting key 348842f499850eee676388020c5fc9ec
I20260926 22:48:33.320248 3651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:33.320573 3651 log.cc:824] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:33.321285 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a: No bootstrap required, opened a new log
I20260926 22:48:33.323284 3651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45241 } }
I20260926 22:48:33.323448 3651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:33.323516 3651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14cc4602a43649e69cb7fd93dd9cb66a, State: Initialized, Role: FOLLOWER
I20260926 22:48:33.323642 3651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45241 } }
I20260926 22:48:33.323706 3651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:33.323726 3651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:33.323791 3651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:33.324432 3651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45241 } }
I20260926 22:48:33.324548 3651 leader_election.cc:302] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14cc4602a43649e69cb7fd93dd9cb66a; no voters:
I20260926 22:48:33.324836 3651 leader_election.cc:288] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:33.324847 3656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:33.324980 3656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [term 1 LEADER]: Becoming Leader. State: Replica: 14cc4602a43649e69cb7fd93dd9cb66a, State: Running, Role: LEADER
I20260926 22:48:33.325058 3656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45241 } }
I20260926 22:48:33.325098 3651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:33.325395 3658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14cc4602a43649e69cb7fd93dd9cb66a. Latest consensus state: current_term: 1 leader_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45241 } } }
I20260926 22:48:33.325487 3658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:33.325644 3657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cc4602a43649e69cb7fd93dd9cb66a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45241 } } }
I20260926 22:48:33.325686 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:33.320281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:36843 (local address 127.0.0.1:45241)
0926 22:48:33.320453 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:33.320460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:33.320483 (+ 23us) server_negotiation.cc:407] Connection header received
0926 22:48:33.320937 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:33.320944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:33.320992 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:33.321102 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:33.321360 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:33.322224 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:33.323394 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:33.324006 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:33.324085 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:33.324806 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:33.324810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:33.324811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:33.325027 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:33.325065 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:33.325079 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:33.325179 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:33.325440 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 22:48:33.325466 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260926 22:48:33.325745 3662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:33.326238 3662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:33.326841 3657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:33.326861 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:33.327895 3662 catalog_manager.cc:1347] Generated new cluster ID: 34a097959c3a45889beff3ae0164cd16
I20260926 22:48:33.327989 3662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:33.331688 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:33.328286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:52545 (local address 127.0.0.1:45241)
0926 22:48:33.328321 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:33.328323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:33.328460 (+ 137us) server_negotiation.cc:407] Connection header received
0926 22:48:33.328503 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:33.328504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:33.328512 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:33.328557 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:33.328701 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:33.329390 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:33.330425 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:33.330754 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:33.330820 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:33.331453 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:33.331455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:33.331455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:33.331459 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:33.331471 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:33.331475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:33.331528 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:33.331613 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 22:48:33.331622 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:48:33.347643 3662 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:33.348178 3662 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:33.351135 3662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14cc4602a43649e69cb7fd93dd9cb66a: Generated new TSK 0
I20260926 22:48:33.351321 3662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:33.357218 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3583
W20260926 22:48:33.361333 3568 connection.cc:535] client connection to 127.0.0.1:45241 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45241): Connection reset by peer (error 104)
2026-09-26T22:48:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:33 dist-test-slave-3986 krb5kdc[3682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:33 dist-test-slave-3986 krb5kdc[3682](info): set up 2 sockets
Sep 26 22:48:33 dist-test-slave-3986 krb5kdc[3682](info): commencing operation
krb5kdc: starting...
W20260926 22:48:34.387053 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:34 dist-test-slave-3986 krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:34.501097 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:34.581547 3695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:34.581748 3695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:34.581769 3695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:34.582880 3695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:34.582913 3695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:34.582927 3695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:34.582940 3695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:34.582952 3695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:34.584568 3695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:34.584807 3695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:34.585044 3695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:34.589804 3701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:34.589928 3695 server_base.cc:1034] running on GCE node
W20260926 22:48:34.589761 3703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:34.589761 3700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:34.590256 3695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:34.590531 3695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:34.591703 3695 hybrid_clock.cc:648] HybridClock initialized: now 1790462914591687 us; error 35 us; skew 500 ppm
Sep 26 22:48:34 dist-test-slave-3986 krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462914, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:34.594161 3695 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:34.594650 3695 webserver.cc:460] Webserver started at http://127.0.0.1:33841/ using document root <none> and password file <none>
I20260926 22:48:34.594945 3695 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:34.594991 3695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:34.595206 3695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:34.596246 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "03809b01724c4c02a130be76913367a3"
format_stamp: "Formatted at 2026-09-26 22:48:34 on dist-test-slave-3986"
server_key: "86c7b0b2ea8a7349fe1d8ba5e522d429"
server_key_iv: "0c7415c5a81ca939dbcf18a6dad13e24"
server_key_version: "encryptionkey@0"
I20260926 22:48:34.596603 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "03809b01724c4c02a130be76913367a3"
format_stamp: "Formatted at 2026-09-26 22:48:34 on dist-test-slave-3986"
server_key: "86c7b0b2ea8a7349fe1d8ba5e522d429"
server_key_iv: "0c7415c5a81ca939dbcf18a6dad13e24"
server_key_version: "encryptionkey@0"
I20260926 22:48:34.597883 3695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:34.598600 3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:34.598788 3695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:34.598894 3695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "03809b01724c4c02a130be76913367a3"
format_stamp: "Formatted at 2026-09-26 22:48:34 on dist-test-slave-3986"
server_key: "86c7b0b2ea8a7349fe1d8ba5e522d429"
server_key_iv: "0c7415c5a81ca939dbcf18a6dad13e24"
server_key_version: "encryptionkey@0"
I20260926 22:48:34.598974 3695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:34.602121 3695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:34.603951 3695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:34.604100 3695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:34.608073 3695 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36185
I20260926 22:48:34.608105 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36185 every 8 connection(s)
I20260926 22:48:34.608412 3695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:34.609200 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:34.611389 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3: Bootstrap starting.
I20260926 22:48:34.611912 3763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:34.612135 3763 log.cc:824] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:34.612727 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3: No bootstrap required, opened a new log
I20260926 22:48:34.614229 3763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03809b01724c4c02a130be76913367a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36185 } }
I20260926 22:48:34.614362 3763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:34.614384 3763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03809b01724c4c02a130be76913367a3, State: Initialized, Role: FOLLOWER
I20260926 22:48:34.614514 3763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03809b01724c4c02a130be76913367a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36185 } }
I20260926 22:48:34.614595 3763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:34.614617 3763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:34.614714 3763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:34.615209 3763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03809b01724c4c02a130be76913367a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36185 } }
I20260926 22:48:34.615336 3763 leader_election.cc:302] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03809b01724c4c02a130be76913367a3; no voters:
I20260926 22:48:34.615506 3763 leader_election.cc:288] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:34.615484 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3695
I20260926 22:48:34.615567 3766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:34.615603 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:34.615715 3766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [term 1 LEADER]: Becoming Leader. State: Replica: 03809b01724c4c02a130be76913367a3, State: Running, Role: LEADER
I20260926 22:48:34.615768 3763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:34.615842 15075 external_mini_cluster.cc:1428] Setting key aced9a98c0a05963d437a18fcf08fe03
I20260926 22:48:34.615833 3766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03809b01724c4c02a130be76913367a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36185 } }
I20260926 22:48:34.616326 3767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03809b01724c4c02a130be76913367a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03809b01724c4c02a130be76913367a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36185 } } }
I20260926 22:48:34.616416 3767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:34.616770 3768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03809b01724c4c02a130be76913367a3. Latest consensus state: current_term: 1 leader_uuid: "03809b01724c4c02a130be76913367a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03809b01724c4c02a130be76913367a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36185 } } }
I20260926 22:48:34.616915 3768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:34.617087 3776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:34.617635 3776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:34.618757 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:34.616604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:37655 (local address 127.0.0.1:36185)
0926 22:48:34.616862 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:34.616872 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:34.616910 (+ 38us) server_negotiation.cc:407] Connection header received
0926 22:48:34.617031 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:34.617038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:34.617068 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:34.617481 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:34.617940 (+ 459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:34.617943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:34.617944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:34.617947 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:34.617986 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:34.618000 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:34.618065 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:34.618163 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 22:48:34.618204 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260926 22:48:34.619262 3776 catalog_manager.cc:1347] Generated new cluster ID: 0fe7af10f45745eea0e2e5439f0d7a19
I20260926 22:48:34.619314 3776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:34.619674 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:34.622083 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:34.621009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:36619 (local address 127.0.0.1:36185)
0926 22:48:34.621042 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:34.621044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:34.621164 (+ 120us) server_negotiation.cc:407] Connection header received
0926 22:48:34.621237 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:34.621238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:34.621243 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:34.621297 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:34.621863 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:34.621864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:34.621864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:34.621869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:34.621883 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:34.621887 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:34.621943 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:34.621990 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 22:48:34.622004 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:34.662696 3776 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:34.662909 3776 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:34.672459 3776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03809b01724c4c02a130be76913367a3: Generated new TSK 0
I20260926 22:48:34.672655 3776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:34.684341 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3695
2026-09-26T22:48:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:34 dist-test-slave-3986 krb5kdc[3794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:34 dist-test-slave-3986 krb5kdc[3794](info): set up 2 sockets
Sep 26 22:48:34 dist-test-slave-3986 krb5kdc[3794](info): commencing operation
krb5kdc: starting...
W20260926 22:48:35.703154 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:35 dist-test-slave-3986 krb5kdc[3794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:35.813575 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:35.892150 3807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:35.892346 3807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:35.892367 3807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:35.893486 3807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:35.893529 3807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:35.893543 3807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:35.893580 3807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:35.893610 3807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:35.895133 3807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:35.895378 3807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:35.895613 3807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:35.900418 3815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:35.900445 3807 server_base.cc:1034] running on GCE node
W20260926 22:48:35.900372 3812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:35.900373 3813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:35.900847 3807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:35.901152 3807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:35.902313 3807 hybrid_clock.cc:648] HybridClock initialized: now 1790462915902293 us; error 42 us; skew 500 ppm
Sep 26 22:48:35 dist-test-slave-3986 krb5kdc[3794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462915, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:35.904778 3807 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:35.905205 3807 webserver.cc:460] Webserver started at http://127.0.0.1:46559/ using document root <none> and password file <none>
I20260926 22:48:35.905441 3807 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:35.905488 3807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:35.905647 3807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:35.906744 3807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "8780954c6a40484a96f7bbe28e78aab4"
format_stamp: "Formatted at 2026-09-26 22:48:35 on dist-test-slave-3986"
server_key: "bfa2318e2e37019915a28b75aa8afcb7"
server_key_iv: "9752fde79d2ae7f94e1182320a7c40c1"
server_key_version: "encryptionkey@0"
I20260926 22:48:35.907097 3807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "8780954c6a40484a96f7bbe28e78aab4"
format_stamp: "Formatted at 2026-09-26 22:48:35 on dist-test-slave-3986"
server_key: "bfa2318e2e37019915a28b75aa8afcb7"
server_key_iv: "9752fde79d2ae7f94e1182320a7c40c1"
server_key_version: "encryptionkey@0"
I20260926 22:48:35.908428 3807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:35.909235 3822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:35.909415 3807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:35.909521 3807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "8780954c6a40484a96f7bbe28e78aab4"
format_stamp: "Formatted at 2026-09-26 22:48:35 on dist-test-slave-3986"
server_key: "bfa2318e2e37019915a28b75aa8afcb7"
server_key_iv: "9752fde79d2ae7f94e1182320a7c40c1"
server_key_version: "encryptionkey@0"
I20260926 22:48:35.909593 3807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:35.923636 3807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:35.925371 3807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:35.925535 3807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:35.929406 3807 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37335
I20260926 22:48:35.929461 3874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37335 every 8 connection(s)
I20260926 22:48:35.929816 3807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:35.930817 3875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:35.932974 3875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4: Bootstrap starting.
I20260926 22:48:35.933550 3875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:35.933831 3875 log.cc:824] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:35.934506 3875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4: No bootstrap required, opened a new log
I20260926 22:48:35.936071 3875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780954c6a40484a96f7bbe28e78aab4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260926 22:48:35.936201 3875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:35.936224 3875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8780954c6a40484a96f7bbe28e78aab4, State: Initialized, Role: FOLLOWER
I20260926 22:48:35.936419 3875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780954c6a40484a96f7bbe28e78aab4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260926 22:48:35.936498 3875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:35.936550 3875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:35.936600 3875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:35.937103 3875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780954c6a40484a96f7bbe28e78aab4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260926 22:48:35.937209 3875 leader_election.cc:302] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8780954c6a40484a96f7bbe28e78aab4; no voters:
I20260926 22:48:35.937393 3875 leader_election.cc:288] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:35.937453 3878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:35.937790 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3807
I20260926 22:48:35.937896 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:35.938097 15075 external_mini_cluster.cc:1428] Setting key 95881ba4041d2bb33f88a15f80a0d69d
I20260926 22:48:35.937614 3878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [term 1 LEADER]: Becoming Leader. State: Replica: 8780954c6a40484a96f7bbe28e78aab4, State: Running, Role: LEADER
I20260926 22:48:35.937661 3875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:35.938330 3878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780954c6a40484a96f7bbe28e78aab4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260926 22:48:35.938944 3880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8780954c6a40484a96f7bbe28e78aab4. Latest consensus state: current_term: 1 leader_uuid: "8780954c6a40484a96f7bbe28e78aab4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780954c6a40484a96f7bbe28e78aab4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260926 22:48:35.939023 3879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8780954c6a40484a96f7bbe28e78aab4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780954c6a40484a96f7bbe28e78aab4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260926 22:48:35.939033 3880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:35.939363 3879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4 [sys.catalog]: This master's current role is: LEADER
W20260926 22:48:35.940307 3896 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:35.940373 3896 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:35.940425 3890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:35.940586 3890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:35.942133 3890 catalog_manager.cc:1347] Generated new cluster ID: 20029175a6aa436c9b7aa5ba054410ab
I20260926 22:48:35.942188 3890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:35 dist-test-slave-3986 krb5kdc[3794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 22:48:35.949868 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:35.938541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:33887 (local address 127.0.0.1:37335)
0926 22:48:35.938781 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:35.938790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:35.939002 (+ 212us) server_negotiation.cc:407] Connection header received
0926 22:48:35.939222 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:35.939229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:35.939317 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:35.939436 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:35.940461 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:35.941700 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:35.942870 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:35.943460 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:35.943525 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:35.946298 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:35.946301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:35.946302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:35.946312 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:35.948209 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:35.948619 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:35.948622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:35.948622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:35.948713 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:35.948915 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:35.948916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:35.948916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:35.949181 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:35.949307 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:35.949557 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 22:48:35.949654 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":61,"threads_started":1}
I20260926 22:48:35.951112 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:35.953241 3890 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:35.953940 3890 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:35.958725 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:35.952688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:44765 (local address 127.0.0.1:37335)
0926 22:48:35.952719 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:35.952721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:35.952747 (+ 26us) server_negotiation.cc:407] Connection header received
0926 22:48:35.952811 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:35.952812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:35.952822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:35.952876 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:35.953511 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:35.954007 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:35.954766 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:35.954986 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:35.955069 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:35.956140 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:35.956142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:35.956142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:35.956148 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:35.957545 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:35.957912 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:35.957913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:35.957913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:35.957965 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:35.958191 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:35.958192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:35.958192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:35.958299 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:35.958422 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:35.958535 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 22:48:35.958595 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 22:48:35.960505 3890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8780954c6a40484a96f7bbe28e78aab4: Generated new TSK 0
I20260926 22:48:35.960669 3890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:35.965973 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3807
W20260926 22:48:35.969791 3792 connection.cc:535] client connection to 127.0.0.1:37335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37335): Connection reset by peer (error 104)
2026-09-26T22:48:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:36 dist-test-slave-3986 krb5kdc[3906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:36 dist-test-slave-3986 krb5kdc[3906](info): set up 2 sockets
Sep 26 22:48:36 dist-test-slave-3986 krb5kdc[3906](info): commencing operation
krb5kdc: starting...
W20260926 22:48:36.990876 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:37 dist-test-slave-3986 krb5kdc[3906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:37.101255 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:37.180552 3919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:37.180799 3919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:37.180837 3919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:37.181985 3919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:37.182066 3919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:37.182088 3919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:37.182108 3919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:37.182138 3919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:37.183606 3919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:37.183902 3919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:37.184134 3919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:37.188915 3925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:37.188938 3927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:37.188975 3919 server_base.cc:1034] running on GCE node
W20260926 22:48:37.188938 3924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:37.189332 3919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:37.189630 3919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:37.190811 3919 hybrid_clock.cc:648] HybridClock initialized: now 1790462917190805 us; error 35 us; skew 500 ppm
Sep 26 22:48:37 dist-test-slave-3986 krb5kdc[3906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:37.196089 3919 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:37.196516 3919 webserver.cc:460] Webserver started at http://127.0.0.1:45623/ using document root <none> and password file <none>
I20260926 22:48:37.196709 3919 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:37.196750 3919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:37.196834 3919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:37.197757 3919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "96cc3469d1fd416bbed035793855bf1b"
format_stamp: "Formatted at 2026-09-26 22:48:37 on dist-test-slave-3986"
server_key: "de1ed447e8f93cd46cecf26d02aa925c"
server_key_iv: "5ca738929f4d2e584e38cf6b272f5a46"
server_key_version: "encryptionkey@0"
I20260926 22:48:37.198038 3919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "96cc3469d1fd416bbed035793855bf1b"
format_stamp: "Formatted at 2026-09-26 22:48:37 on dist-test-slave-3986"
server_key: "de1ed447e8f93cd46cecf26d02aa925c"
server_key_iv: "5ca738929f4d2e584e38cf6b272f5a46"
server_key_version: "encryptionkey@0"
I20260926 22:48:37.199369 3919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 22:48:37.200062 3934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:37.200253 3919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:37.200337 3919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "96cc3469d1fd416bbed035793855bf1b"
format_stamp: "Formatted at 2026-09-26 22:48:37 on dist-test-slave-3986"
server_key: "de1ed447e8f93cd46cecf26d02aa925c"
server_key_iv: "5ca738929f4d2e584e38cf6b272f5a46"
server_key_version: "encryptionkey@0"
I20260926 22:48:37.200390 3919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:37.213917 3919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:37.215651 3919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:37.215756 3919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:37.219383 3919 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34459
I20260926 22:48:37.219437 3986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34459 every 8 connection(s)
I20260926 22:48:37.219756 3919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:37.220611 3987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:37.222822 3987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b: Bootstrap starting.
I20260926 22:48:37.223354 3987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:37.223630 3987 log.cc:824] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:37.224328 3987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b: No bootstrap required, opened a new log
I20260926 22:48:37.225335 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 3919
I20260926 22:48:37.225447 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:37.225639 15075 external_mini_cluster.cc:1428] Setting key f434fe6dc2d316fe46c6d8472880b876
I20260926 22:48:37.226390 3987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc3469d1fd416bbed035793855bf1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260926 22:48:37.226522 3987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:37.226543 3987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cc3469d1fd416bbed035793855bf1b, State: Initialized, Role: FOLLOWER
I20260926 22:48:37.226727 3987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc3469d1fd416bbed035793855bf1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260926 22:48:37.226818 3987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:37.226861 3987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:37.226913 3987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:37.227443 3987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc3469d1fd416bbed035793855bf1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260926 22:48:37.227526 3987 leader_election.cc:302] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96cc3469d1fd416bbed035793855bf1b; no voters:
I20260926 22:48:37.227721 3987 leader_election.cc:288] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:37.227746 3992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:37.227854 3992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [term 1 LEADER]: Becoming Leader. State: Replica: 96cc3469d1fd416bbed035793855bf1b, State: Running, Role: LEADER
I20260926 22:48:37.227979 3987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:37.227988 3992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc3469d1fd416bbed035793855bf1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } }
I20260926 22:48:37.228426 3994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96cc3469d1fd416bbed035793855bf1b. Latest consensus state: current_term: 1 leader_uuid: "96cc3469d1fd416bbed035793855bf1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc3469d1fd416bbed035793855bf1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } } }
I20260926 22:48:37.228520 3994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:37.228706 3993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96cc3469d1fd416bbed035793855bf1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc3469d1fd416bbed035793855bf1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34459 } } }
I20260926 22:48:37.229041 3993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b [sys.catalog]: This master's current role is: LEADER
W20260926 22:48:37.229874 4008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:37.229928 4008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:37.229975 3998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:37.230119 3998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:37.231587 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:37.226151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:53225 (local address 127.0.0.1:34459)
0926 22:48:37.226371 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:37.226378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:37.226401 (+ 23us) server_negotiation.cc:407] Connection header received
0926 22:48:37.226485 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:37.226490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:37.226562 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:37.226712 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:37.226958 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:37.228390 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:37.229475 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:37.230060 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:37.230164 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:37.230869 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:37.230872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:37.230873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:37.231084 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:37.231121 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:37.231134 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:37.231241 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:37.231321 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 22:48:37.231347 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":113,"threads_started":1}
I20260926 22:48:37.231648 3998 catalog_manager.cc:1347] Generated new cluster ID: 45e2aef2dcef4b018cdc43bb14f1dc1a
I20260926 22:48:37.231757 3998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:37.232662 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:37.237514 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:37.233919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:48515 (local address 127.0.0.1:34459)
0926 22:48:37.233955 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:37.233957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:37.234931 (+ 974us) server_negotiation.cc:407] Connection header received
0926 22:48:37.235002 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:37.235003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:37.235014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:37.235069 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:37.235205 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:37.235664 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:37.236378 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:37.236595 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:37.236666 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:37.237300 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:37.237301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:37.237302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:37.237306 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:37.237319 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:37.237322 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:37.237374 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:37.237432 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 22:48:37.237442 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:37.246975 3998 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:37.247473 3998 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:37.250185 3998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96cc3469d1fd416bbed035793855bf1b: Generated new TSK 0
I20260926 22:48:37.250380 3998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:37.254398 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 3919
W20260926 22:48:37.258492 3904 connection.cc:535] client connection to 127.0.0.1:34459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34459): Connection reset by peer (error 104)
2026-09-26T22:48:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:37 dist-test-slave-3986 krb5kdc[4018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:37 dist-test-slave-3986 krb5kdc[4018](info): set up 2 sockets
Sep 26 22:48:37 dist-test-slave-3986 krb5kdc[4018](info): commencing operation
krb5kdc: starting...
W20260926 22:48:38.279019 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:38 dist-test-slave-3986 krb5kdc[4018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 22:48:38.390949 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46423
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:38.473526 4031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:38.473752 4031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:38.473789 4031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:38.475006 4031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:38.475066 4031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:38.475091 4031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:38.475123 4031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:38.475147 4031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:38.476629 4031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46423
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:38.476939 4031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:38.477154 4031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:38.481776 4039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:38.481882 4036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:38.481930 4037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:38.482182 4031 server_base.cc:1034] running on GCE node
I20260926 22:48:38.482388 4031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:38.482612 4031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:38.483747 4031 hybrid_clock.cc:648] HybridClock initialized: now 1790462918483733 us; error 28 us; skew 500 ppm
Sep 26 22:48:38 dist-test-slave-3986 krb5kdc[4018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:38.486047 4031 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 22:48:38.486413 4031 webserver.cc:460] Webserver started at http://127.0.0.1:41229/ using document root <none> and password file <none>
I20260926 22:48:38.486585 4031 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:38.486624 4031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:38.486788 4031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:38.487779 4031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "469a23393f42481aa56d93fea48498e0"
format_stamp: "Formatted at 2026-09-26 22:48:38 on dist-test-slave-3986"
server_key: "c1e52b83bc6075f73aa74c6696d8eafb"
server_key_iv: "5aace16645a95cec3c98fc892bdbd7d0"
server_key_version: "encryptionkey@0"
I20260926 22:48:38.488077 4031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "469a23393f42481aa56d93fea48498e0"
format_stamp: "Formatted at 2026-09-26 22:48:38 on dist-test-slave-3986"
server_key: "c1e52b83bc6075f73aa74c6696d8eafb"
server_key_iv: "5aace16645a95cec3c98fc892bdbd7d0"
server_key_version: "encryptionkey@0"
I20260926 22:48:38.489295 4031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 22:48:38.489917 4046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:38.490078 4031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:38.490182 4031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "469a23393f42481aa56d93fea48498e0"
format_stamp: "Formatted at 2026-09-26 22:48:38 on dist-test-slave-3986"
server_key: "c1e52b83bc6075f73aa74c6696d8eafb"
server_key_iv: "5aace16645a95cec3c98fc892bdbd7d0"
server_key_version: "encryptionkey@0"
I20260926 22:48:38.490260 4031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:38.493512 4031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:38.495739 4031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:38.495903 4031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:38.499694 4031 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46423
I20260926 22:48:38.499733 4098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46423 every 8 connection(s)
I20260926 22:48:38.500070 4031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:38.500916 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:38.503077 4099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0: Bootstrap starting.
I20260926 22:48:38.503577 4099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:38.503808 4099 log.cc:824] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:38.504405 4099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0: No bootstrap required, opened a new log
I20260926 22:48:38.505028 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4031
I20260926 22:48:38.505137 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:38.505340 15075 external_mini_cluster.cc:1428] Setting key ebcf01a9964a5fdd108d664cbcf2c0d1
I20260926 22:48:38.506552 4099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469a23393f42481aa56d93fea48498e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46423 } }
I20260926 22:48:38.506743 4099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:38.506846 4099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469a23393f42481aa56d93fea48498e0, State: Initialized, Role: FOLLOWER
I20260926 22:48:38.506992 4099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469a23393f42481aa56d93fea48498e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46423 } }
I20260926 22:48:38.507117 4099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:38.507159 4099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:38.507215 4099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:38.507836 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:38.505872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:36493 (local address 127.0.0.1:46423)
0926 22:48:38.506038 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:38.506045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:38.506069 (+ 24us) server_negotiation.cc:407] Connection header received
0926 22:48:38.506532 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:38.506539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:38.506571 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:38.506778 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:38.507366 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:38.507370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:38.507370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:38.507374 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:38.507410 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:38.507421 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:38.507477 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:38.507574 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 22:48:38.507626 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260926 22:48:38.508299 4099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469a23393f42481aa56d93fea48498e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46423 } }
I20260926 22:48:38.508411 4099 leader_election.cc:302] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 469a23393f42481aa56d93fea48498e0; no voters:
I20260926 22:48:38.508571 4099 leader_election.cc:288] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:38.508615 4104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:38.508756 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:38.508752 4104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [term 1 LEADER]: Becoming Leader. State: Replica: 469a23393f42481aa56d93fea48498e0, State: Running, Role: LEADER
I20260926 22:48:38.508818 4099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:38.508857 4104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469a23393f42481aa56d93fea48498e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46423 } }
I20260926 22:48:38.509137 4104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469a23393f42481aa56d93fea48498e0. Latest consensus state: current_term: 1 leader_uuid: "469a23393f42481aa56d93fea48498e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469a23393f42481aa56d93fea48498e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46423 } } }
I20260926 22:48:38.509227 4104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:38.509150 4105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "469a23393f42481aa56d93fea48498e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469a23393f42481aa56d93fea48498e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46423 } } }
I20260926 22:48:38.509266 4105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:38.509413 4110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:38.509912 4110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:38.511693 4110 catalog_manager.cc:1347] Generated new cluster ID: 83922441a7ba44d18c69f1e4a07ebe7f
I20260926 22:48:38.511745 4110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:38.515898 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:38.511468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.55:60147 (local address 127.0.0.1:46423)
0926 22:48:38.511501 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:38.511503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:38.514906 (+ 3403us) server_negotiation.cc:407] Connection header received
0926 22:48:38.514990 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:38.514991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:38.514997 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:38.515048 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:38.515686 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:38.515687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:38.515688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:38.515693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:38.515708 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:38.515712 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:38.515757 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:38.515817 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 22:48:38.515831 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:48:38.540884 4110 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:38.541075 4110 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:38.544319 4110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 469a23393f42481aa56d93fea48498e0: Generated new TSK 0
I20260926 22:48:38.544488 4110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:38.549202 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4031
2026-09-26T22:48:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:38 dist-test-slave-3986 krb5kdc[4130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:38 dist-test-slave-3986 krb5kdc[4130](info): set up 2 sockets
Sep 26 22:48:38 dist-test-slave-3986 krb5kdc[4130](info): commencing operation
krb5kdc: starting...
W20260926 22:48:39.579018 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:39 dist-test-slave-3986 krb5kdc[4130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:39.695202 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45171
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:41207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:45171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:39.774773 4143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:39.775036 4143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:39.775079 4143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:39.776239 4143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:39.776319 4143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:39.776345 4143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:39.776372 4143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:39.776404 4143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:39.777866 4143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:41207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:45171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:45171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:39.778144 4143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:39.778376 4143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:39.783161 4148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:39.783208 4149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:39.783253 4143 server_base.cc:1034] running on GCE node
W20260926 22:48:39.783209 4151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:39.783651 4143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:39.783939 4143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:39.785112 4143 hybrid_clock.cc:648] HybridClock initialized: now 1790462919785091 us; error 36 us; skew 500 ppm
Sep 26 22:48:39 dist-test-slave-3986 krb5kdc[4130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:39.787694 4143 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:39.788079 4143 webserver.cc:460] Webserver started at http://127.14.184.254:43749/ using document root <none> and password file <none>
I20260926 22:48:39.788283 4143 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:39.788326 4143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:39.788410 4143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:39.789501 4143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "4569b74db8204013ba8dc1fbfc6a3b75"
format_stamp: "Formatted at 2026-09-26 22:48:39 on dist-test-slave-3986"
server_key: "e9c7944c981c3af18f9e4fdcff8170ae"
server_key_iv: "6efbc8ce2d02a699962dad5b4151c745"
server_key_version: "encryptionkey@0"
I20260926 22:48:39.789801 4143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "4569b74db8204013ba8dc1fbfc6a3b75"
format_stamp: "Formatted at 2026-09-26 22:48:39 on dist-test-slave-3986"
server_key: "e9c7944c981c3af18f9e4fdcff8170ae"
server_key_iv: "6efbc8ce2d02a699962dad5b4151c745"
server_key_version: "encryptionkey@0"
I20260926 22:48:39.791105 4143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:39.791783 4158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:39.791968 4143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:39.792039 4143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "4569b74db8204013ba8dc1fbfc6a3b75"
format_stamp: "Formatted at 2026-09-26 22:48:39 on dist-test-slave-3986"
server_key: "e9c7944c981c3af18f9e4fdcff8170ae"
server_key_iv: "6efbc8ce2d02a699962dad5b4151c745"
server_key_version: "encryptionkey@0"
I20260926 22:48:39.792092 4143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:39.813027 4143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:39.815440 4143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:39.815551 4143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:39.819370 4143 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:45171
I20260926 22:48:39.819427 4210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:45171 every 8 connection(s)
I20260926 22:48:39.819762 4143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:39.820591 4211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:39.822891 4211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75: Bootstrap starting.
I20260926 22:48:39.823450 4211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:39.823734 4211 log.cc:824] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:39.824432 4211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75: No bootstrap required, opened a new log
I20260926 22:48:39.825976 4211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45171 } }
I20260926 22:48:39.826146 4211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:39.826190 4211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4569b74db8204013ba8dc1fbfc6a3b75, State: Initialized, Role: FOLLOWER
I20260926 22:48:39.826375 4211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45171 } }
I20260926 22:48:39.826476 4211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:39.826520 4211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:39.826574 4211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:39.827407 4211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45171 } }
I20260926 22:48:39.827526 4211 leader_election.cc:302] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4569b74db8204013ba8dc1fbfc6a3b75; no voters:
I20260926 22:48:39.827677 4211 leader_election.cc:288] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:39.827710 4214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:39.827884 4214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [term 1 LEADER]: Becoming Leader. State: Replica: 4569b74db8204013ba8dc1fbfc6a3b75, State: Running, Role: LEADER
I20260926 22:48:39.827940 4211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:39.828090 4214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45171 } }
I20260926 22:48:39.828475 4215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45171 } } }
I20260926 22:48:39.828570 4215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:39.828480 4216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4569b74db8204013ba8dc1fbfc6a3b75. Latest consensus state: current_term: 1 leader_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4569b74db8204013ba8dc1fbfc6a3b75" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 45171 } } }
I20260926 22:48:39.828704 4216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:39.829054 4223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:39.829493 4223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:39.829779 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4143
I20260926 22:48:39.829902 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:39.830101 15075 external_mini_cluster.cc:1428] Setting key c3edbe66b23610dba5b465f6d5ab5a84
I20260926 22:48:39.831095 4223 catalog_manager.cc:1347] Generated new cluster ID: e56cb4149ee6439ea88ca92972847169
I20260926 22:48:39.831144 4223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:39 dist-test-slave-3986 krb5kdc[4130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:39.839850 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:39.830751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46915 (local address 127.14.184.254:45171)
0926 22:48:39.831139 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:39.831145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:39.831171 (+ 26us) server_negotiation.cc:407] Connection header received
0926 22:48:39.831265 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:39.831270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:39.831309 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:39.831448 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:39.832068 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:39.832932 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:39.833701 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:39.834105 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:39.834249 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:39.836523 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:39.836527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:39.836528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:39.836538 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:39.838168 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:39.838575 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:39.838580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:39.838581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:39.838732 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:39.838976 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:39.838976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:39.838977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:39.839209 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:39.839399 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:39.839489 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 22:48:39.839608 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":72,"threads_started":1}
I20260926 22:48:39.841043 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:39.848603 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:39.842390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37017 (local address 127.14.184.254:45171)
0926 22:48:39.842425 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:39.842426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:39.842545 (+ 119us) server_negotiation.cc:407] Connection header received
0926 22:48:39.842617 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:39.842617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:39.842627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:39.842700 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:39.843214 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:39.843720 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:39.844428 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:39.844662 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:39.844736 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:39.845963 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:39.845965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:39.845965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:39.845973 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:39.847403 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:39.847777 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:39.847778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:39.847778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:39.847832 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:39.848064 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:39.848065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:39.848065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:39.848177 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:39.848314 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:39.848422 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 22:48:39.848485 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:39.855051 4223 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:39.855556 4223 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:39.864027 4223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4569b74db8204013ba8dc1fbfc6a3b75: Generated new TSK 0
I20260926 22:48:39.864238 4223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:39.867965 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4143
W20260926 22:48:39.872167 4128 connection.cc:535] client connection to 127.14.184.254:45171 recv error: Network error: failed to read from TLS socket (remote: 127.14.184.254:45171): Connection reset by peer (error 104)
2026-09-26T22:48:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:39 dist-test-slave-3986 krb5kdc[4242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:39 dist-test-slave-3986 krb5kdc[4242](info): set up 2 sockets
Sep 26 22:48:39 dist-test-slave-3986 krb5kdc[4242](info): commencing operation
krb5kdc: starting...
W20260926 22:48:40.899019 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:40 dist-test-slave-3986 krb5kdc[4242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:41.006836 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33795
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:33795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:41.085167 4255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:41.085423 4255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:41.085460 4255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:41.086604 4255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:41.086673 4255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:41.086723 4255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:41.086773 4255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:41.086804 4255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:41.088279 4255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:33795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:41.088551 4255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:41.088797 4255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:41.094033 4260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:41.094033 4261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:41.094117 4263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:41.094383 4255 server_base.cc:1034] running on GCE node
I20260926 22:48:41.094651 4255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:41.094980 4255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:41.096150 4255 hybrid_clock.cc:648] HybridClock initialized: now 1790462921096136 us; error 28 us; skew 500 ppm
Sep 26 22:48:41 dist-test-slave-3986 krb5kdc[4242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462921, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:41.098524 4255 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:41.098942 4255 webserver.cc:460] Webserver started at http://127.14.184.254:34131/ using document root <none> and password file <none>
I20260926 22:48:41.099130 4255 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:41.099169 4255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:41.099252 4255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:41.100188 4255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "06ab25b5515e45b4ba5229c0b7042b2d"
format_stamp: "Formatted at 2026-09-26 22:48:41 on dist-test-slave-3986"
server_key: "75db2075db64d5dd916bc54cf19b21ca"
server_key_iv: "bb524d6ec1688d89caa2d546afd21071"
server_key_version: "encryptionkey@0"
I20260926 22:48:41.100492 4255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "06ab25b5515e45b4ba5229c0b7042b2d"
format_stamp: "Formatted at 2026-09-26 22:48:41 on dist-test-slave-3986"
server_key: "75db2075db64d5dd916bc54cf19b21ca"
server_key_iv: "bb524d6ec1688d89caa2d546afd21071"
server_key_version: "encryptionkey@0"
I20260926 22:48:41.101711 4255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:41.102368 4270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:41.102561 4255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:41.102699 4255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "06ab25b5515e45b4ba5229c0b7042b2d"
format_stamp: "Formatted at 2026-09-26 22:48:41 on dist-test-slave-3986"
server_key: "75db2075db64d5dd916bc54cf19b21ca"
server_key_iv: "bb524d6ec1688d89caa2d546afd21071"
server_key_version: "encryptionkey@0"
I20260926 22:48:41.102769 4255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:41.111599 4255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:41.113312 4255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:41.113454 4255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:41.117158 4255 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:33795
I20260926 22:48:41.117187 4322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:33795 every 8 connection(s)
I20260926 22:48:41.117513 4255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:41.118340 4323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:41.120559 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4255
I20260926 22:48:41.120564 4323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d: Bootstrap starting.
I20260926 22:48:41.120667 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:41.120858 15075 external_mini_cluster.cc:1428] Setting key 5ff10a5ff14efff7bb41ef66dbb10be0
I20260926 22:48:41.121230 4323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:41.121567 4323 log.cc:824] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:41.122215 4323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d: No bootstrap required, opened a new log
I20260926 22:48:41.124015 4323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33795 } }
I20260926 22:48:41.124161 4323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:41.124208 4323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ab25b5515e45b4ba5229c0b7042b2d, State: Initialized, Role: FOLLOWER
I20260926 22:48:41.124336 4323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33795 } }
I20260926 22:48:41.124423 4323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:41.124466 4323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:41.124513 4323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:41.125000 4323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33795 } }
I20260926 22:48:41.125131 4323 leader_election.cc:302] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06ab25b5515e45b4ba5229c0b7042b2d; no voters:
I20260926 22:48:41.125288 4323 leader_election.cc:288] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:41.125337 4328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:41.125473 4328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [term 1 LEADER]: Becoming Leader. State: Replica: 06ab25b5515e45b4ba5229c0b7042b2d, State: Running, Role: LEADER
I20260926 22:48:41.125552 4323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:41.125612 4328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33795 } }
I20260926 22:48:41.126027 4329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33795 } } }
I20260926 22:48:41.126113 4329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:41.126322 4330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06ab25b5515e45b4ba5229c0b7042b2d. Latest consensus state: current_term: 1 leader_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ab25b5515e45b4ba5229c0b7042b2d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 33795 } } }
I20260926 22:48:41.126659 4330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:41.126361 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:41.121368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46219 (local address 127.14.184.254:33795)
0926 22:48:41.121565 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:41.121574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:41.121605 (+ 31us) server_negotiation.cc:407] Connection header received
0926 22:48:41.122064 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:41.122070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:41.122111 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:41.122258 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:41.122444 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:41.123317 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:41.124100 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:41.124686 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:41.124799 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:41.125511 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:41.125515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:41.125516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:41.125729 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:41.125765 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:41.125779 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:41.125853 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:41.126129 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 22:48:41.126157 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260926 22:48:41.126899 4337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:41.127565 4337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:41.127712 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:41.129002 4337 catalog_manager.cc:1347] Generated new cluster ID: 4814e5e753a74e7f82341a23c4f797fb
I20260926 22:48:41.129056 4337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:41.132808 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:41.129303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57261 (local address 127.14.184.254:33795)
0926 22:48:41.129331 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:41.129332 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:41.129358 (+ 26us) server_negotiation.cc:407] Connection header received
0926 22:48:41.129439 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:41.129439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:41.129447 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:41.129497 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:41.129649 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:41.130366 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:41.131522 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:41.131844 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:41.131907 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:41.132559 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:41.132560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:41.132560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:41.132564 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:41.132576 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:41.132580 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:41.132633 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:41.132731 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 22:48:41.132741 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260926 22:48:41.154874 4337 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:41.155404 4337 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:41.159699 4337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06ab25b5515e45b4ba5229c0b7042b2d: Generated new TSK 0
I20260926 22:48:41.159897 4337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:41.170228 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4255
W20260926 22:48:41.174520 4240 connection.cc:535] client connection to 127.14.184.254:33795 recv error: Network error: failed to read from TLS socket (remote: 127.14.184.254:33795): Connection reset by peer (error 104)
2026-09-26T22:48:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:41 dist-test-slave-3986 krb5kdc[4354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:41 dist-test-slave-3986 krb5kdc[4354](info): set up 2 sockets
Sep 26 22:48:41 dist-test-slave-3986 krb5kdc[4354](info): commencing operation
krb5kdc: starting...
W20260926 22:48:42.186954 15075 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:42 dist-test-slave-3986 krb5kdc[4354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:42.297869 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:41731
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:41731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:42.375274 4367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:42.375495 4367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:42.375532 4367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:42.376664 4367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:42.376740 4367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:42.376765 4367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:42.376824 4367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:42.376856 4367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:42.378279 4367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:41731
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:41731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:42.378542 4367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:42.378814 4367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:42.383472 4375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:42.383473 4373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:42.383559 4372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:42.383945 4367 server_base.cc:1034] running on GCE node
I20260926 22:48:42.384202 4367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:42.384512 4367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:42.385653 4367 hybrid_clock.cc:648] HybridClock initialized: now 1790462922385642 us; error 27 us; skew 500 ppm
Sep 26 22:48:42 dist-test-slave-3986 krb5kdc[4354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462922, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:42.391996 4367 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:42.392410 4367 webserver.cc:460] Webserver started at http://127.14.184.254:36847/ using document root <none> and password file <none>
I20260926 22:48:42.392628 4367 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:42.392695 4367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:42.392848 4367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:42.393883 4367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "3ca263901bcc4f14a128893ddb2332c2"
format_stamp: "Formatted at 2026-09-26 22:48:42 on dist-test-slave-3986"
server_key: "ddb14646bfca4c9eb1eb8d576c5543bd"
server_key_iv: "f354461808587141a8e664af4c32c838"
server_key_version: "encryptionkey@0"
I20260926 22:48:42.394232 4367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "3ca263901bcc4f14a128893ddb2332c2"
format_stamp: "Formatted at 2026-09-26 22:48:42 on dist-test-slave-3986"
server_key: "ddb14646bfca4c9eb1eb8d576c5543bd"
server_key_iv: "f354461808587141a8e664af4c32c838"
server_key_version: "encryptionkey@0"
I20260926 22:48:42.395622 4367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:42.396327 4382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:42.396513 4367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:42.396598 4367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "3ca263901bcc4f14a128893ddb2332c2"
format_stamp: "Formatted at 2026-09-26 22:48:42 on dist-test-slave-3986"
server_key: "ddb14646bfca4c9eb1eb8d576c5543bd"
server_key_iv: "f354461808587141a8e664af4c32c838"
server_key_version: "encryptionkey@0"
I20260926 22:48:42.396651 4367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:42.399770 4367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:42.401852 4367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:42.401955 4367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:42.405560 4367 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:41731
I20260926 22:48:42.405622 4434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:41731 every 8 connection(s)
I20260926 22:48:42.405915 4367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:42.406888 4435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:42.408938 4435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2: Bootstrap starting.
I20260926 22:48:42.409471 4435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:42.409762 4435 log.cc:824] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:42.410387 4435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2: No bootstrap required, opened a new log
I20260926 22:48:42.411701 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4367
I20260926 22:48:42.411821 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:42.412007 15075 external_mini_cluster.cc:1428] Setting key f79b6c6c95e066b49bc1a77d467f6997
I20260926 22:48:42.412437 4435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca263901bcc4f14a128893ddb2332c2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41731 } }
I20260926 22:48:42.412670 4435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:42.412714 4435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca263901bcc4f14a128893ddb2332c2, State: Initialized, Role: FOLLOWER
I20260926 22:48:42.412842 4435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca263901bcc4f14a128893ddb2332c2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41731 } }
I20260926 22:48:42.412902 4435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:42.412923 4435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:42.412990 4435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:42.413498 4435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca263901bcc4f14a128893ddb2332c2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41731 } }
I20260926 22:48:42.413580 4435 leader_election.cc:302] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ca263901bcc4f14a128893ddb2332c2; no voters:
I20260926 22:48:42.413805 4435 leader_election.cc:288] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:42.413862 4440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:42.414064 4435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:42.414197 4440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [term 1 LEADER]: Becoming Leader. State: Replica: 3ca263901bcc4f14a128893ddb2332c2, State: Running, Role: LEADER
I20260926 22:48:42.414431 4440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca263901bcc4f14a128893ddb2332c2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41731 } }
I20260926 22:48:42.414656 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:42.412487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34347 (local address 127.14.184.254:41731)
0926 22:48:42.412678 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:42.412687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:42.412740 (+ 53us) server_negotiation.cc:407] Connection header received
0926 22:48:42.412857 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:42.412864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:42.412913 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:42.413044 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:42.414066 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:42.414070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:42.414070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:42.414074 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:42.414120 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:42.414136 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:42.414201 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:42.414382 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 22:48:42.414422 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260926 22:48:42.414970 4442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ca263901bcc4f14a128893ddb2332c2. Latest consensus state: current_term: 1 leader_uuid: "3ca263901bcc4f14a128893ddb2332c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca263901bcc4f14a128893ddb2332c2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41731 } } }
I20260926 22:48:42.415056 4442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:42.415278 4449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:42.415465 4441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ca263901bcc4f14a128893ddb2332c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca263901bcc4f14a128893ddb2332c2" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 41731 } } }
I20260926 22:48:42.415664 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:42.415783 4441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:42.415827 4449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:42.418005 4449 catalog_manager.cc:1347] Generated new cluster ID: c462e7f9daea4b16904a100111a9517e
I20260926 22:48:42.418066 4449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:42.418325 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:42.417044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54315 (local address 127.14.184.254:41731)
0926 22:48:42.417075 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:42.417077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:42.417325 (+ 248us) server_negotiation.cc:407] Connection header received
0926 22:48:42.417387 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:42.417388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:42.417392 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:42.417462 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:42.418053 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:42.418055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:42.418055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:42.418059 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:42.418072 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:42.418077 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:42.418123 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:42.418249 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 22:48:42.418261 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:42.436738 4449 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:42.436910 4449 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:42.439634 4449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ca263901bcc4f14a128893ddb2332c2: Generated new TSK 0
I20260926 22:48:42.439822 4449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:42.444779 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4367
2026-09-26T22:48:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:42 dist-test-slave-3986 krb5kdc[4466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:42 dist-test-slave-3986 krb5kdc[4466](info): set up 2 sockets
Sep 26 22:48:42 dist-test-slave-3986 krb5kdc[4466](info): commencing operation
krb5kdc: starting...
W20260926 22:48:43.466993 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:43 dist-test-slave-3986 krb5kdc[4466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:43.581987 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:36263
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:36263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:43.660025 4479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:43.660218 4479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:43.660239 4479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:43.661286 4479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:43.661320 4479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:43.661334 4479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:43.661346 4479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:43.661358 4479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:43.662747 4479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:36263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:36263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:43.662930 4479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:43.663156 4479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:43.667939 4484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:43.667917 4485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:43.667927 4487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:43.668265 4479 server_base.cc:1034] running on GCE node
I20260926 22:48:43.668504 4479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:43.668830 4479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:43.669992 4479 hybrid_clock.cc:648] HybridClock initialized: now 1790462923669975 us; error 48 us; skew 500 ppm
Sep 26 22:48:43 dist-test-slave-3986 krb5kdc[4466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462923, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:43.672374 4479 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:43.672799 4479 webserver.cc:460] Webserver started at http://127.14.184.254:41281/ using document root <none> and password file <none>
I20260926 22:48:43.673017 4479 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:43.673081 4479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:43.673239 4479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:43.674242 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "907a29b469da4baf8f8d8d921fc8b77d"
format_stamp: "Formatted at 2026-09-26 22:48:43 on dist-test-slave-3986"
server_key: "d8ce280dfbd6cf0a3718047aa3dc3346"
server_key_iv: "23a1650d2882d11913da2a7884e6df48"
server_key_version: "encryptionkey@0"
I20260926 22:48:43.674590 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "907a29b469da4baf8f8d8d921fc8b77d"
format_stamp: "Formatted at 2026-09-26 22:48:43 on dist-test-slave-3986"
server_key: "d8ce280dfbd6cf0a3718047aa3dc3346"
server_key_iv: "23a1650d2882d11913da2a7884e6df48"
server_key_version: "encryptionkey@0"
I20260926 22:48:43.676015 4479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:43.676798 4494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:43.676973 4479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:43.677073 4479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "907a29b469da4baf8f8d8d921fc8b77d"
format_stamp: "Formatted at 2026-09-26 22:48:43 on dist-test-slave-3986"
server_key: "d8ce280dfbd6cf0a3718047aa3dc3346"
server_key_iv: "23a1650d2882d11913da2a7884e6df48"
server_key_version: "encryptionkey@0"
I20260926 22:48:43.677160 4479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:43.692705 4479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:43.694423 4479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:43.694584 4479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:43.698279 4479 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:36263
I20260926 22:48:43.698316 4546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:36263 every 8 connection(s)
I20260926 22:48:43.698653 4479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:43.699486 4547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:43.701673 4547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d: Bootstrap starting.
I20260926 22:48:43.702239 4547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:43.702515 4547 log.cc:824] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:43.703244 4547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d: No bootstrap required, opened a new log
I20260926 22:48:43.704775 4547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907a29b469da4baf8f8d8d921fc8b77d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36263 } }
I20260926 22:48:43.704913 4547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:43.704936 4547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907a29b469da4baf8f8d8d921fc8b77d, State: Initialized, Role: FOLLOWER
I20260926 22:48:43.705112 4547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907a29b469da4baf8f8d8d921fc8b77d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36263 } }
I20260926 22:48:43.705189 4547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:43.705212 4547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:43.705276 4547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:43.706063 4547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907a29b469da4baf8f8d8d921fc8b77d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36263 } }
I20260926 22:48:43.706146 4547 leader_election.cc:302] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907a29b469da4baf8f8d8d921fc8b77d; no voters:
I20260926 22:48:43.706313 4547 leader_election.cc:288] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:43.706350 4550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:43.706470 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4479
I20260926 22:48:43.706564 4550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [term 1 LEADER]: Becoming Leader. State: Replica: 907a29b469da4baf8f8d8d921fc8b77d, State: Running, Role: LEADER
I20260926 22:48:43.706583 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:43.706584 4547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:43.706760 4550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907a29b469da4baf8f8d8d921fc8b77d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36263 } }
I20260926 22:48:43.706792 15075 external_mini_cluster.cc:1428] Setting key f2e40227d1fce5201d322e5089f6196c
I20260926 22:48:43.707166 4551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "907a29b469da4baf8f8d8d921fc8b77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907a29b469da4baf8f8d8d921fc8b77d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36263 } } }
I20260926 22:48:43.707255 4551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:43.707424 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907a29b469da4baf8f8d8d921fc8b77d. Latest consensus state: current_term: 1 leader_uuid: "907a29b469da4baf8f8d8d921fc8b77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907a29b469da4baf8f8d8d921fc8b77d" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 36263 } } }
I20260926 22:48:43.707515 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:43.707726 4560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:43.708442 4560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:43.710397 4560 catalog_manager.cc:1347] Generated new cluster ID: 83b5d9ee4d024ee8ba36971d3b0b347e
I20260926 22:48:43.710445 4560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 22:48:43 dist-test-slave-3986 krb5kdc[4466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:43.718906 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:43.707364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34129 (local address 127.14.184.254:36263)
0926 22:48:43.707526 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:43.707535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:43.707809 (+ 274us) server_negotiation.cc:407] Connection header received
0926 22:48:43.707929 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:43.707936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:43.707994 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:43.708127 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:43.708963 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:43.710011 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:43.711117 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:43.711713 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:43.711806 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:43.714514 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:43.714519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:43.714519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:43.714531 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:43.717139 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:43.717526 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:43.717529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:43.717530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:43.717662 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:43.717864 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:43.717865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:43.717865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:43.718177 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:43.718352 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:43.718421 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 22:48:43.718686 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260926 22:48:43.720082 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:43.723047 4560 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:43.723511 4560 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:43.728809 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:43.721521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48055 (local address 127.14.184.254:36263)
0926 22:48:43.721554 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:43.721555 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:43.721676 (+ 121us) server_negotiation.cc:407] Connection header received
0926 22:48:43.721767 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:43.721768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:43.721778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:43.721807 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:43.722471 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:43.723244 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:43.724327 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:43.724656 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:43.724710 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:43.726117 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:43.726119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:43.726119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:43.726126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:43.727618 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:43.727990 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:43.727991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:43.727992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:43.728043 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:43.728229 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:43.728230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:43.728230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:43.728358 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:43.728435 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:43.728639 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 22:48:43.728724 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 22:48:43.730058 4560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 907a29b469da4baf8f8d8d921fc8b77d: Generated new TSK 0
I20260926 22:48:43.730214 4560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:43.733817 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4479
2026-09-26T22:48:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:43 dist-test-slave-3986 krb5kdc[4578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:43 dist-test-slave-3986 krb5kdc[4578](info): set up 2 sockets
Sep 26 22:48:43 dist-test-slave-3986 krb5kdc[4578](info): commencing operation
krb5kdc: starting...
W20260926 22:48:44.759048 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:44 dist-test-slave-3986 krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:44.868999 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43651
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:43651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:44.947109 4591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:44.947330 4591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:44.947368 4591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:44.948495 4591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:44.948559 4591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:44.948585 4591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:44.948616 4591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:44.948639 4591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:44.950083 4591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:43651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:43651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:44.950350 4591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:44.950567 4591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:44.955343 4596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:44.955318 4599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:44.955451 4591 server_base.cc:1034] running on GCE node
W20260926 22:48:44.955312 4597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:44.955713 4591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:44.956024 4591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:44.957166 4591 hybrid_clock.cc:648] HybridClock initialized: now 1790462924957145 us; error 35 us; skew 500 ppm
Sep 26 22:48:44 dist-test-slave-3986 krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462924, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:44.959523 4591 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:44.959966 4591 webserver.cc:460] Webserver started at http://127.14.184.254:45795/ using document root <none> and password file <none>
I20260926 22:48:44.960189 4591 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:44.960232 4591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:44.960397 4591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:44.961409 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "f5b7c1acd51e4e9c88604abcc58362c7"
format_stamp: "Formatted at 2026-09-26 22:48:44 on dist-test-slave-3986"
server_key: "28d937ba51f70a95da0ab7d4bed16908"
server_key_iv: "c00b851136a7486e38caeb4d5d6044b5"
server_key_version: "encryptionkey@0"
I20260926 22:48:44.961763 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "f5b7c1acd51e4e9c88604abcc58362c7"
format_stamp: "Formatted at 2026-09-26 22:48:44 on dist-test-slave-3986"
server_key: "28d937ba51f70a95da0ab7d4bed16908"
server_key_iv: "c00b851136a7486e38caeb4d5d6044b5"
server_key_version: "encryptionkey@0"
I20260926 22:48:44.963061 4591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:44.963745 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:44.963941 4591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:44.964030 4591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "f5b7c1acd51e4e9c88604abcc58362c7"
format_stamp: "Formatted at 2026-09-26 22:48:44 on dist-test-slave-3986"
server_key: "28d937ba51f70a95da0ab7d4bed16908"
server_key_iv: "c00b851136a7486e38caeb4d5d6044b5"
server_key_version: "encryptionkey@0"
I20260926 22:48:44.964102 4591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:44.983839 4591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:44.985737 4591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:44.985895 4591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:44.989941 4591 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:43651
I20260926 22:48:44.990016 4658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:43651 every 8 connection(s)
I20260926 22:48:44.990337 4591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:44.991233 4659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:44.993263 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4591
I20260926 22:48:44.993391 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:44.993559 4659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7: Bootstrap starting.
I20260926 22:48:44.993610 15075 external_mini_cluster.cc:1428] Setting key 02f31d907bdd20bff0209dfe94fb4322
I20260926 22:48:44.994271 4659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:44.994593 4659 log.cc:824] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:44.995543 4659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7: No bootstrap required, opened a new log
I20260926 22:48:44.997572 4659 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43651 } }
I20260926 22:48:44.997732 4659 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:44.997762 4659 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b7c1acd51e4e9c88604abcc58362c7, State: Initialized, Role: FOLLOWER
I20260926 22:48:44.997902 4659 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43651 } }
I20260926 22:48:44.997959 4659 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:44.997980 4659 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:44.998054 4659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:44.998798 4659 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43651 } }
I20260926 22:48:44.998910 4659 leader_election.cc:302] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5b7c1acd51e4e9c88604abcc58362c7; no voters:
I20260926 22:48:44.999073 4659 leader_election.cc:288] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:44.999091 4664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:44.999204 4664 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [term 1 LEADER]: Becoming Leader. State: Replica: f5b7c1acd51e4e9c88604abcc58362c7, State: Running, Role: LEADER
I20260926 22:48:44.999284 4664 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43651 } }
I20260926 22:48:44.999336 4659 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:44.999588 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43651 } } }
I20260926 22:48:44.999675 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:44.999868 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:44.994094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38363 (local address 127.14.184.254:43651)
0926 22:48:44.994293 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:44.994308 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:44.994357 (+ 49us) server_negotiation.cc:407] Connection header received
0926 22:48:44.995048 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:44.995057 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:44.995125 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:44.995276 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:44.995510 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:44.996434 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:44.997611 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:44.998178 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:44.998261 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:44.998943 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:44.998947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:44.998947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:44.999164 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:44.999205 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:44.999221 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:44.999554 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:44.999587 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 22:48:44.999652 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260926 22:48:44.999853 4666 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5b7c1acd51e4e9c88604abcc58362c7. Latest consensus state: current_term: 1 leader_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b7c1acd51e4e9c88604abcc58362c7" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 43651 } } }
I20260926 22:48:44.999915 4666 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:45.000808 4669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:45.000847 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:45.001312 4669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:45.002621 4669 catalog_manager.cc:1347] Generated new cluster ID: cf1de6ef0df942b8b4e688b3f68d32eb
I20260926 22:48:45.002693 4669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:45.010532 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:45.007108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52705 (local address 127.14.184.254:43651)
0926 22:48:45.007149 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:45.007151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:45.007201 (+ 50us) server_negotiation.cc:407] Connection header received
0926 22:48:45.007258 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:45.007259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:45.007270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:45.007346 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:45.007481 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:45.008229 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:45.009212 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:45.009498 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:45.009587 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:45.010264 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:45.010265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:45.010265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:45.010270 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:45.010288 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:45.010292 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:45.010351 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:45.010407 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 22:48:45.010432 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:48:45.017024 4669 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:45.017541 4669 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:45.024505 4669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5b7c1acd51e4e9c88604abcc58362c7: Generated new TSK 0
I20260926 22:48:45.024694 4669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:45.029799 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4591
2026-09-26T22:48:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:45 dist-test-slave-3986 krb5kdc[4690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:45 dist-test-slave-3986 krb5kdc[4690](info): set up 2 sockets
Sep 26 22:48:45 dist-test-slave-3986 krb5kdc[4690](info): commencing operation
krb5kdc: starting...
W20260926 22:48:46.047091 15075 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:46 dist-test-slave-3986 krb5kdc[4690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:46.157368 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34619
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:34619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:46.235783 4703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:46.236019 4703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:46.236055 4703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:46.237195 4703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:46.237268 4703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:46.237295 4703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:46.237327 4703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:46.237351 4703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:46.238847 4703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:34619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:46.239107 4703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:46.239324 4703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:46.243927 4709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:46.243969 4711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:46.243937 4708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:46.244004 4703 server_base.cc:1034] running on GCE node
I20260926 22:48:46.244346 4703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:46.244606 4703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:46.245740 4703 hybrid_clock.cc:648] HybridClock initialized: now 1790462926245731 us; error 31 us; skew 500 ppm
Sep 26 22:48:46 dist-test-slave-3986 krb5kdc[4690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462926, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:46.248070 4703 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:46.248530 4703 webserver.cc:460] Webserver started at http://127.14.184.254:41173/ using document root <none> and password file <none>
I20260926 22:48:46.248759 4703 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:46.248826 4703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:46.248970 4703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:46.249965 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "a498ef09baa94d25aeb53c138295c1a5"
format_stamp: "Formatted at 2026-09-26 22:48:46 on dist-test-slave-3986"
server_key: "9917b2c1467a096c6112fcff146da7f9"
server_key_iv: "d98c0838cde34c4289eaf504fad42ee1"
server_key_version: "encryptionkey@0"
I20260926 22:48:46.250321 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "a498ef09baa94d25aeb53c138295c1a5"
format_stamp: "Formatted at 2026-09-26 22:48:46 on dist-test-slave-3986"
server_key: "9917b2c1467a096c6112fcff146da7f9"
server_key_iv: "d98c0838cde34c4289eaf504fad42ee1"
server_key_version: "encryptionkey@0"
I20260926 22:48:46.251642 4703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:46.252339 4718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:46.252504 4703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:46.252615 4703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "a498ef09baa94d25aeb53c138295c1a5"
format_stamp: "Formatted at 2026-09-26 22:48:46 on dist-test-slave-3986"
server_key: "9917b2c1467a096c6112fcff146da7f9"
server_key_iv: "d98c0838cde34c4289eaf504fad42ee1"
server_key_version: "encryptionkey@0"
I20260926 22:48:46.252687 4703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:46.256101 4703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:46.257862 4703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:46.258018 4703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:46.261996 4703 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:34619
I20260926 22:48:46.262024 4770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:34619 every 8 connection(s)
I20260926 22:48:46.262352 4703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:46.263314 4771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:46.265386 4771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5: Bootstrap starting.
I20260926 22:48:46.265919 4771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:46.266188 4771 log.cc:824] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:46.266880 4771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5: No bootstrap required, opened a new log
I20260926 22:48:46.268385 4771 raft_consensus.cc:348] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a498ef09baa94d25aeb53c138295c1a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34619 } }
I20260926 22:48:46.268550 4771 raft_consensus.cc:374] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 22:48:46.268599 4771 raft_consensus.cc:729] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a498ef09baa94d25aeb53c138295c1a5, State: Initialized, Role: FOLLOWER
I20260926 22:48:46.268780 4771 consensus_queue.cc:260] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a498ef09baa94d25aeb53c138295c1a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34619 } }
I20260926 22:48:46.268864 4771 raft_consensus.cc:388] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 22:48:46.268913 4771 raft_consensus.cc:482] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 22:48:46.268960 4771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:46.269726 4771 raft_consensus.cc:504] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a498ef09baa94d25aeb53c138295c1a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34619 } }
I20260926 22:48:46.269842 4771 leader_election.cc:302] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a498ef09baa94d25aeb53c138295c1a5; no voters:
I20260926 22:48:46.270015 4771 leader_election.cc:288] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 22:48:46.270071 4774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:46.270213 4774 raft_consensus.cc:686] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [term 1 LEADER]: Becoming Leader. State: Replica: a498ef09baa94d25aeb53c138295c1a5, State: Running, Role: LEADER
I20260926 22:48:46.270253 4771 sys_catalog.cc:564] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:46.270320 4774 consensus_queue.cc:237] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a498ef09baa94d25aeb53c138295c1a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34619 } }
I20260926 22:48:46.270735 4775 sys_catalog.cc:455] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a498ef09baa94d25aeb53c138295c1a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a498ef09baa94d25aeb53c138295c1a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34619 } } }
I20260926 22:48:46.270748 4776 sys_catalog.cc:455] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a498ef09baa94d25aeb53c138295c1a5. Latest consensus state: current_term: 1 leader_uuid: "a498ef09baa94d25aeb53c138295c1a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a498ef09baa94d25aeb53c138295c1a5" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34619 } } }
I20260926 22:48:46.270848 4775 sys_catalog.cc:458] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:46.270856 4776 sys_catalog.cc:458] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:46.271068 4783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:46.271417 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4703
I20260926 22:48:46.271536 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:46.271569 4783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:46.271724 15075 external_mini_cluster.cc:1428] Setting key b33d98eb6c5023464b38d6d53e478dd3
I20260926 22:48:46.273483 4783 catalog_manager.cc:1347] Generated new cluster ID: 03849a5be2b24667901a9f26f4e3ae0d
I20260926 22:48:46.273550 4783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:46.274070 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:46.272331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37131 (local address 127.14.184.254:34619)
0926 22:48:46.272479 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:46.272487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:46.272571 (+ 84us) server_negotiation.cc:407] Connection header received
0926 22:48:46.272698 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:46.272705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:46.272735 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:46.272890 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:46.273630 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:46.273634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:46.273634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:46.273638 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:46.273677 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:46.273697 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:46.273771 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:46.273846 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 22:48:46.273900 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260926 22:48:46.274958 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:46.277326 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:46.276262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44889 (local address 127.14.184.254:34619)
0926 22:48:46.276297 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:46.276298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:46.276421 (+ 123us) server_negotiation.cc:407] Connection header received
0926 22:48:46.276483 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:46.276484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:46.276489 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:46.276544 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:46.277117 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:46.277118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:46.277118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 22:48:46.277123 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:46.277135 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 22:48:46.277139 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:46.277189 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:46.277240 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 22:48:46.277255 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:46.298014 4783 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:46.298182 4783 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:46.303946 4783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a498ef09baa94d25aeb53c138295c1a5: Generated new TSK 0
I20260926 22:48:46.304119 4783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 22:48:46.316377 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4703
2026-09-26T22:48:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1286 ms)
[----------] 18 tests from AuthTokenIssuingTest (23391 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:46 dist-test-slave-3986 krb5kdc[4802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:46 dist-test-slave-3986 krb5kdc[4802](info): set up 2 sockets
Sep 26 22:48:46 dist-test-slave-3986 krb5kdc[4802](info): commencing operation
krb5kdc: starting...
W20260926 22:48:47.335106 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:47.445241 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34557
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:34557,127.14.184.253:41391,127.14.184.252:36489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:47.525012 4815 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 22:48:47.525249 4815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:47.525288 4815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:47.525329 4815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:47.526473 4815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:47.526537 4815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:47.526561 4815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:47.526594 4815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:47.526619 4815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:47.528106 4815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:34557,127.14.184.253:41391,127.14.184.252:36489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:34557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:47.528370 4815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:47.528578 4815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:47.533382 4823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:47.533390 4820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:47.533675 4821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:47.533830 4815 server_base.cc:1034] running on GCE node
I20260926 22:48:47.534076 4815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:47.534305 4815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:47.535466 4815 hybrid_clock.cc:648] HybridClock initialized: now 1790462927535452 us; error 29 us; skew 500 ppm
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:47.543211 4815 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:47.543628 4815 webserver.cc:460] Webserver started at http://127.14.184.254:46131/ using document root <none> and password file <none>
I20260926 22:48:47.543848 4815 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:47.543912 4815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:47.544065 4815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:47.545089 4815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "c8944d188e9356a208aa464de6eef311"
server_key_iv: "2a8c1072eb889988ed18e69617ef56ca"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.545449 4815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "c8944d188e9356a208aa464de6eef311"
server_key_iv: "2a8c1072eb889988ed18e69617ef56ca"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.546710 4815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 22:48:47.547454 4830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:47.547622 4815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 22:48:47.547719 4815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "c8944d188e9356a208aa464de6eef311"
server_key_iv: "2a8c1072eb889988ed18e69617ef56ca"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.547807 4815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:47.572448 4815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:47.574815 4815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:47.574980 4815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:47.578652 4815 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:34557
I20260926 22:48:47.578737 4882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:34557 every 8 connection(s)
I20260926 22:48:47.579129 4815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:47.579700 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4815
I20260926 22:48:47.579834 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:47.580029 15075 external_mini_cluster.cc:1428] Setting key e2be6732a4b97c8822806c67ccc4d93b
I20260926 22:48:47.580207 4883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:47.582264 4883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.184.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.253@KRBTEST.COM" created.
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:47.594965 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.582989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34557 (local address 127.0.0.1:48508)
0926 22:48:47.583157 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.583196 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.583789 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.584073 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.584076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.584435 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.584659 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.584665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.586146 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.586147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.587343 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.587347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.588050 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.588051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.588115 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.588616 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.588624 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.591390 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.593417 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.593423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.593425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.593616 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.593869 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.593870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.593871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.593959 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.594329 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.594334 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.594395 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.594505 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.594756 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":112,"thread_start_us":78,"threads_started":1}
I20260926 22:48:47.594972 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.583011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.14.184.254:34557)
0926 22:48:47.583176 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.583186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.583306 (+ 120us) server_negotiation.cc:407] Connection header received
0926 22:48:47.583938 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.583940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.583959 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.584059 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.584766 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.586043 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.587446 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.587945 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.588066 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.591495 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.591505 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.591505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.591516 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.593325 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.593710 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.593713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.593713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.593801 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.594042 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.594043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.594043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.594258 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.594347 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.594536 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.594699 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260926 22:48:47.595688 4883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:47.596037 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.595924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37004)
0926 22:48:47.595970 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.596015 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 22:48:47.596167 4834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
I20260926 22:48:47.596565 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.596471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37012)
0926 22:48:47.596521 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.596544 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 22:48:47.596666 4883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:41391: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab.
Entry for principal kudu/127.14.184.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab.
I20260926 22:48:47.605813 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:41391
--webserver_interface=127.14.184.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab
--principal=kudu/127.14.184.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:34557,127.14.184.253:41391,127.14.184.252:36489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:48:47.645874 4883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } attempt: 1
I20260926 22:48:47.646437 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.646243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37022)
0926 22:48:47.646346 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.646398 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 22:48:47.646870 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.646766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37026)
0926 22:48:47.646815 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.646845 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 22:48:47.647009 4883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:41391: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
W20260926 22:48:47.686802 4890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 22:48:47.686990 4890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:47.687006 4890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:47.687026 4890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:47.688169 4890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:47.688210 4890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:47.688225 4890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:47.688258 4890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:47.688287 4890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:47.689646 4890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:34557,127.14.184.253:41391,127.14.184.252:36489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab
--principal=kudu/127.14.184.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:41391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.184.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:47.689812 4890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:47.690055 4890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:47.694540 4898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:47.694734 4896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:47.694648 4895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:47.694877 4890 server_base.cc:1034] running on GCE node
I20260926 22:48:47.695112 4890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:47.695389 4890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:47.696550 4890 hybrid_clock.cc:648] HybridClock initialized: now 1790462927696536 us; error 26 us; skew 500 ppm
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:47.698879 4890 init.cc:377] Logged in from keytab as kudu/127.14.184.253@KRBTEST.COM (short username kudu)
I20260926 22:48:47.699272 4890 webserver.cc:460] Webserver started at http://127.14.184.253:43487/ using document root <none> and password file <none>
I20260926 22:48:47.699481 4890 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:47.699548 4890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:47.699693 4890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:47.700666 4890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data/instance:
uuid: "536e4bb4b9f54115b6bf5639993e3b84"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "509415f5ebda422f3150ed489afda531"
server_key_iv: "04c4a548bd1581de52654f766e20f58e"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.701015 4890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal/instance:
uuid: "536e4bb4b9f54115b6bf5639993e3b84"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "509415f5ebda422f3150ed489afda531"
server_key_iv: "04c4a548bd1581de52654f766e20f58e"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.702353 4890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:47.703130 4905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:47.703296 4890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:47.703399 4890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal
uuid: "536e4bb4b9f54115b6bf5639993e3b84"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "509415f5ebda422f3150ed489afda531"
server_key_iv: "04c4a548bd1581de52654f766e20f58e"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.703474 4890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:47.715167 4883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } attempt: 2
I20260926 22:48:47.715570 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.715449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37034)
0926 22:48:47.715507 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.715542 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 22:48:47.715942 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.715857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37038)
0926 22:48:47.715897 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.715921 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:48:47.716043 4883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:41391: Network error: Client connection negotiation failed: client connection to 127.14.184.253:41391: connect: Connection refused (error 111)
I20260926 22:48:47.723146 4890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:47.724856 4890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:47.725008 4890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:47.729097 4890 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.253:41391
I20260926 22:48:47.729161 4957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.253:41391 every 8 connection(s)
I20260926 22:48:47.729462 4890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
I20260926 22:48:47.730278 4958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:47.730559 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4890
I20260926 22:48:47.730664 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-1/wal/instance
I20260926 22:48:47.730863 15075 external_mini_cluster.cc:1428] Setting key 7abe3fdfc1f068051b7ac762b0d78f1b
I20260926 22:48:47.731805 4958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.252@KRBTEST.COM" created.
I20260926 22:48:47.741308 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.732525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.14.184.254:34557)
0926 22:48:47.732563 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.732565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.732771 (+ 206us) server_negotiation.cc:407] Connection header received
0926 22:48:47.733244 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.733246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.733260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.733316 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.733891 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.734399 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.735461 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.735684 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.735777 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.738330 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.738332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.738332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.738340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.739829 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.740309 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.740310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.740311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.740374 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.740629 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.740631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.740631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.740793 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.740865 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.741136 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.741219 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 22:48:47.741560 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.732509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34557 (local address 127.0.0.1:48512)
0926 22:48:47.732654 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.732685 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.733139 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.733380 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.733384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.733612 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.733803 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.733807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.734509 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.734510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.735358 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.735361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.735788 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.735789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.735911 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.736502 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.736511 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.738227 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.739929 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.739937 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.739939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.740215 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.740437 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.740438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.740439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.740564 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.740870 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.740876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.741010 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.741116 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.741270 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260926 22:48:47.742092 4958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:48:47.749583 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.742387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37050)
0926 22:48:47.742437 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.742447 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.742488 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.742765 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.742766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.742920 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.742981 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.742983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.743805 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.743806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.744690 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.744693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.745127 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.745128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.745213 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.745555 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.745560 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.746852 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.748511 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.748512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.748512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.748688 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.748896 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.748897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.748898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.748947 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.749295 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.749296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.749332 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.749394 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.749483 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:48:47.749681 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.742463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.14.184.253:41391)
0926 22:48:47.742598 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.742601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.742615 (+ 14us) server_negotiation.cc:407] Connection header received
0926 22:48:47.742630 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.742631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.742641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.742754 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.743071 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.743709 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.744765 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.745046 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.745103 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.746942 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.746949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.746949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.746956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.748415 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.748778 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.748781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.748781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.748834 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.749032 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.749033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.749034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.749206 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.749315 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.749421 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.749551 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260926 22:48:47.750500 4958 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:47.750916 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.750809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36594)
0926 22:48:47.750849 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.750894 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 22:48:47.751067 4906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
I20260926 22:48:47.751492 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.751395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36596)
0926 22:48:47.751444 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.751477 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 22:48:47.751606 4958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:36489: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab.
Entry for principal kudu/127.14.184.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab.
I20260926 22:48:47.755107 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:36489
--webserver_interface=127.14.184.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab
--principal=kudu/127.14.184.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:34557,127.14.184.253:41391,127.14.184.252:36489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:48:47.800910 4958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } attempt: 1
I20260926 22:48:47.801391 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.801259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36606)
0926 22:48:47.801323 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.801361 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 22:48:47.801748 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.801653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36618)
0926 22:48:47.801697 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.801720 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 22:48:47.801838 4958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:36489: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
I20260926 22:48:47.807199 4883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } attempt: 3
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:48:47.814962 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.807594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.14.184.253:41391)
0926 22:48:47.807655 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.807656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.807666 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:47.807714 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.807715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.807722 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.807772 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.808132 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.808647 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.809682 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.809903 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.809968 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.811952 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.811953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.811954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.811961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.813629 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.814035 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.814037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.814037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.814093 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.814271 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.814271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.814272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.814398 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.814455 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.814748 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.814842 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 22:48:47.815302 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.807544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37064)
0926 22:48:47.807585 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.807598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.807656 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.807790 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.807791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.807976 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.808051 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.808053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.808725 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.808726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.809574 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.809577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.809983 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.809984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.810071 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.810511 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.810517 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.811865 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.813729 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.813730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.813730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.813950 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.814158 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.814158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.814158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.814212 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.814490 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.814491 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.814569 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.815077 (+ 508us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.815188 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:48:47.818750 4883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:47.819157 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.819056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36624)
0926 22:48:47.819103 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.819132 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260926 22:48:47.819950 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.819516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36638)
0926 22:48:47.819896 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.819925 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
W20260926 22:48:47.820077 4883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:36489: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
W20260926 22:48:47.838232 4965 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 22:48:47.838421 4965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:47.838438 4965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:47.838457 4965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:47.839607 4965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:47.839646 4965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:47.839660 4965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:47.839672 4965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:47.839684 4965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:47.841058 4965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:34557,127.14.184.253:41391,127.14.184.252:36489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab
--principal=kudu/127.14.184.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:36489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.184.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:47.841236 4965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:47.841391 4965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:47.845849 4971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:47.845898 4970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:47.846076 4965 server_base.cc:1034] running on GCE node
W20260926 22:48:47.845850 4973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:47.846338 4965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:47.846611 4965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:47.847746 4965 hybrid_clock.cc:648] HybridClock initialized: now 1790462927847736 us; error 26 us; skew 500 ppm
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:47.850097 4965 init.cc:377] Logged in from keytab as kudu/127.14.184.252@KRBTEST.COM (short username kudu)
I20260926 22:48:47.850594 4965 webserver.cc:460] Webserver started at http://127.14.184.252:41119/ using document root <none> and password file <none>
I20260926 22:48:47.850853 4965 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:47.850899 4965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:47.851042 4965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:47.851186 4883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } attempt: 1
I20260926 22:48:47.851553 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.851438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36652)
0926 22:48:47.851493 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.851527 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 22:48:47.851830 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.851749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36658)
0926 22:48:47.851788 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.851815 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 22:48:47.851910 4883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:36489: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
I20260926 22:48:47.852283 4965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data/instance:
uuid: "d51fe638f1894428a627926517b868bc"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "15e011a83ce51d61e616d34e403e4b15"
server_key_iv: "a19fa909b3f60619f0d1aea7ecce0506"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.852641 4965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal/instance:
uuid: "d51fe638f1894428a627926517b868bc"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "15e011a83ce51d61e616d34e403e4b15"
server_key_iv: "a19fa909b3f60619f0d1aea7ecce0506"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.853886 4965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:47.854559 4980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:47.854774 4965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:47.854879 4965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal
uuid: "d51fe638f1894428a627926517b868bc"
format_stamp: "Formatted at 2026-09-26 22:48:47 on dist-test-slave-3986"
server_key: "15e011a83ce51d61e616d34e403e4b15"
server_key_iv: "a19fa909b3f60619f0d1aea7ecce0506"
server_key_version: "encryptionkey@0"
I20260926 22:48:47.854961 4965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:47.869984 4958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } attempt: 2
I20260926 22:48:47.870375 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.870239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36664)
0926 22:48:47.870304 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.870342 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 22:48:47.870779 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.870632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36672)
0926 22:48:47.870715 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.870750 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:48:47.870884 4958 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:36489: Network error: Client connection negotiation failed: client connection to 127.14.184.252:36489: connect: Connection refused (error 111)
I20260926 22:48:47.878894 4965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:47.880643 4965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:47.880805 4965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:47.884654 4965 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.252:36489
I20260926 22:48:47.884684 5032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.252:36489 every 8 connection(s)
I20260926 22:48:47.885018 4965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
I20260926 22:48:47.885828 5033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:47.887382 5033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:47.889617 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 4965
I20260926 22:48:47.889693 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790462668484683-15075-0/minicluster-data/master-2/wal/instance
I20260926 22:48:47.889878 15075 external_mini_cluster.cc:1428] Setting key 3fca3b8216cf374bcc3cf9646a14613f
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:47.896378 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.888067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.14.184.254:34557)
0926 22:48:47.888135 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.888137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.888311 (+ 174us) server_negotiation.cc:407] Connection header received
0926 22:48:47.888733 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.888733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.888742 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.888799 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.889312 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.889808 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.891031 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.891226 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.891304 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.893301 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.893303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.893303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.893318 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.895006 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.895422 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.895423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.895424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.895482 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.895747 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.895748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.895748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.895851 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.895939 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.896248 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.896318 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 22:48:47.896613 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.888040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:34557 (local address 127.0.0.1:48516)
0926 22:48:47.888186 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.888224 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.888658 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.888846 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.888849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.889069 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.889241 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.889244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.889919 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.889920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.890943 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.890946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.891315 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.891316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.891442 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.891906 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.891913 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.893201 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.895082 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.895088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.895089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.895344 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.895552 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.895552 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.895552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.895678 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.895938 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.895943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.896106 (+ 163us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.896225 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.896400 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260926 22:48:47.897168 5033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:47.898592 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.890259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.14.184.252:36489)
0926 22:48:47.890449 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.890453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.890472 (+ 19us) server_negotiation.cc:407] Connection header received
0926 22:48:47.890546 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.890549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.890566 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.890633 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.891314 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.892150 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.892871 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.893216 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.893310 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.895658 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.895663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.895664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.895669 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.897377 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.897771 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.897774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.897774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.897834 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.898030 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.898031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.898031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.898165 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.898333 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.898377 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.898451 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:47.904016 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.897416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:41391 (local address 127.0.0.1:37066)
0926 22:48:47.897456 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.897464 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.897497 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.897624 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.897624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.897725 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.897750 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.897751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.898358 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.898359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.898965 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.898967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.899415 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.899416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.899477 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.899775 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.899780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.900918 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.902491 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.902492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.902492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.902654 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.902890 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.902891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.902891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.902947 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.903715 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.903716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.903758 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.903842 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.903930 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 22:48:47.904023 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.897466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.14.184.253:41391)
0926 22:48:47.897516 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.897518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.897527 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:47.897551 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.897552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.897561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.897619 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.897832 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.898275 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.899046 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.899285 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.899361 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.901003 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.901004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.901004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.901011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.902395 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.902749 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.902750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.902751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.902821 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.903512 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.903512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.903513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.903626 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.903722 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.903868 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.903939 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:48:47.904578 5033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:47.906881 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.899298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.14.184.254:34557)
0926 22:48:47.899332 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.899334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.899359 (+ 25us) server_negotiation.cc:407] Connection header received
0926 22:48:47.899429 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.899430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.899437 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.899472 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.900130 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.900579 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.901378 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.901686 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.901759 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.903899 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.903901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.903901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.903907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.905712 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.906077 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.906078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.906078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.906138 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.906353 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.906354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.906354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.906457 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.906637 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.906670 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.906769 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:47.911162 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.904820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36686)
0926 22:48:47.904849 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.904857 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.904893 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.905097 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.905098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.905233 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.905264 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.905265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.905883 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.905884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.906346 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.906347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.906735 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.906736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.906795 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.907217 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.907222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.908490 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.910132 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.910132 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.910133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.910292 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.910498 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.910498 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.910499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.910549 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.910844 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.910845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.910882 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.910949 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.911037 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260926 22:48:47.911206 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.904863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.14.184.252:36489)
0926 22:48:47.904997 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.904998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.905004 (+ 6us) server_negotiation.cc:407] Connection header received
0926 22:48:47.905019 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.905020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.905027 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.905071 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.905345 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.905795 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.906437 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.906630 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.906719 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.908578 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.908579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.908579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.908585 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.910075 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.910376 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.910377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.910377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.910425 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.910629 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.910630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.910630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.910777 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.910850 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.910974 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.911083 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":118}
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:48:47.912737 5033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: Bootstrap starting.
I20260926 22:48:47.913619 5033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:47.913940 5033 log.cc:824] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:47.914861 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.907502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.14.184.253:41391)
0926 22:48:47.907537 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.907539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.907546 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:48:47.907566 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.907567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.907576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.907641 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.908228 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.908689 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.909648 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.909827 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.909879 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.912160 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.912161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.912161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.912167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.913551 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.913931 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.913932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.913932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.913981 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.914251 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.914252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.914252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.914397 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.914573 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.914638 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.914753 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:47.914881 5033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: No bootstrap required, opened a new log
I20260926 22:48:47.916296 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:47.916706 5033 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.916826 5033 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51fe638f1894428a627926517b868bc, State: Initialized, Role: FOLLOWER
I20260926 22:48:47.916908 5033 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.917203 5033 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:47.917471 4857 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48526
I20260926 22:48:47.918443 5039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.918713 5039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: This master's current role is: FOLLOWER
W20260926 22:48:47.919168 5050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:47.919219 5050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:47.919255 4965 master_runner.cc:186] Error getting master registration for 127.14.184.254:34557: OK, instance_id { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" instance_seqno: 1790462927577661 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 22:48:47.921180 4965 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 22:48:47.927034 4883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } attempt: 2
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:47.933892 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.927288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36702)
0926 22:48:47.927332 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.927342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.927383 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.927514 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.927514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.927659 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.927693 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.927694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.928296 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.928297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.928791 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.928792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.929152 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.929153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.929208 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.929623 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.929628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.931044 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.932809 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.932810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.932811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.933029 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.933230 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.933231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.933231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.933304 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.933581 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.933582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.933629 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.933701 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.933785 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260926 22:48:47.933903 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.927346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.14.184.252:36489)
0926 22:48:47.927406 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.927408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.927418 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:47.927438 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.927438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.927448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.927496 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.927772 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.928215 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.928884 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.929079 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.929154 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.931156 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.931158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.931158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.931165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.932729 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.933106 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.933107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.933107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.933162 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.933367 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.933368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.933368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.933515 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.933578 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.933734 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.933817 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:48:47.935364 4883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37: Bootstrap starting.
I20260926 22:48:47.936041 4883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:47.936326 4883 log.cc:824] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:47.937129 4883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37: No bootstrap required, opened a new log
I20260926 22:48:47.938087 4883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.938189 4883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c7af9fb2ed4741a4a5bf5809ee2e37, State: Initialized, Role: FOLLOWER
I20260926 22:48:47.938288 4883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.938581 5053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.938642 4883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:47.938658 5053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:47.938766 4857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 22:48:47.938882 4857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:47.939654 4857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.940166 4857 leader_election.cc:288] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [CANDIDATE]: Term 1 election: Requested vote from peers 536e4bb4b9f54115b6bf5639993e3b84 (127.14.184.253:41391), d51fe638f1894428a627926517b868bc (127.14.184.252:36489)
W20260926 22:48:47.940423 5064 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:47.940476 5064 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:47.941413 4815 master_runner.cc:186] Error getting master registration for 127.14.184.253:41391: OK, instance_id { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" instance_seqno: 1790462927727982 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 22:48:47.943807 4815 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 22:48:47.943967 5007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d51fe638f1894428a627926517b868bc"
I20260926 22:48:47.944113 5007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:47.945003 5007 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10c7af9fb2ed4741a4a5bf5809ee2e37 in term 1.
I20260926 22:48:47.945231 4831 leader_election.cc:302] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10c7af9fb2ed4741a4a5bf5809ee2e37, d51fe638f1894428a627926517b868bc; no voters:
I20260926 22:48:47.945385 5053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:47.945528 5053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [term 1 LEADER]: Becoming Leader. State: Replica: 10c7af9fb2ed4741a4a5bf5809ee2e37, State: Running, Role: LEADER
I20260926 22:48:47.945616 5053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.946012 5053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c7af9fb2ed4741a4a5bf5809ee2e37. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.946101 5053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:47.952953 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.941482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.14.184.253:41391)
0926 22:48:47.941536 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.941538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.943113 (+ 1575us) server_negotiation.cc:407] Connection header received
0926 22:48:47.943239 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.943240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.943250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.943289 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.944467 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.945156 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.946534 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.947397 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.947473 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.949550 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.949551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.949552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.949558 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.951402 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.951836 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.951838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.951838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.951890 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.952069 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.952070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.952071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.952181 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.952293 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.952482 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.952557 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:47.954818 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.942085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.14.184.252:36489)
0926 22:48:47.942935 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.942937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.946773 (+ 3836us) server_negotiation.cc:407] Connection header received
0926 22:48:47.946879 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.946881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.946892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.946924 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.947718 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.948407 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.949459 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.949772 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.949824 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.951829 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.951830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.951830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.951837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.953564 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.953948 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.953950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.953950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.954002 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.954209 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.954210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.954210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.954337 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.954401 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.954591 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.954730 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832}
I20260926 22:48:47.957604 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.942781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.14.184.254:34557)
0926 22:48:47.942826 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.942828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.946627 (+ 3799us) server_negotiation.cc:407] Connection header received
0926 22:48:47.946715 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.946716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.946726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.946796 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.951371 (+ 4575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.952017 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.953041 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.953343 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.953380 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.954821 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.954822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.954823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.954829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.956444 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.956830 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.956832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.956832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.956892 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.957100 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.957100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.957101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.957219 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.957328 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.957448 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.957518 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 22:48:47.962026 4958 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } attempt: 3
I20260926 22:48:47.962772 5075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:47.962985 5075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:47.964797 5007 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [term 1 FOLLOWER]: Refusing update from remote peer 10c7af9fb2ed4741a4a5bf5809ee2e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:47.965161 5073 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Sep 26 22:48:47 dist-test-slave-3986 krb5kdc[4802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462927, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:47.966398 5039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c7af9fb2ed4741a4a5bf5809ee2e37. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.966490 5039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:47.969565 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.962243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:36489 (local address 127.0.0.1:36728)
0926 22:48:47.962284 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:47.962293 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:47.962326 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:47.962467 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:47.962468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:47.962618 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:47.962657 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.962659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.963457 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.963458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.964246 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:47.964248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.964735 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.964736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:47.964792 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.965276 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:47.965299 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:47.966757 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:47.968538 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.968539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.968540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.968734 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.968964 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:47.968965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:47.968965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:47.969027 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:47.969273 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:47.969274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:47.969317 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 22:48:47.969385 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 22:48:47.969473 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 22:48:47.969887 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:47.962281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36728 (local address 127.14.184.252:36489)
0926 22:48:47.962331 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:47.962333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:47.962374 (+ 41us) server_negotiation.cc:407] Connection header received
0926 22:48:47.962392 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:47.962392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:47.962401 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:47.962450 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:47.962753 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.963373 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.964342 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:47.964630 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:47.964688 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:47.966861 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:47.966862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:47.966862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:47.966869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:47.968448 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.968818 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.968819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.968819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.968896 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:47.969094 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:47.969095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:47.969095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:47.969209 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:47.969271 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:47.969725 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 22:48:47.969793 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 22:48:47.971380 4958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: Bootstrap starting.
I20260926 22:48:47.972090 4958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:47.972379 4958 log.cc:824] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:47.973951 4958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: No bootstrap required, opened a new log
I20260926 22:48:47.975586 5053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.975673 5053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:47.975728 5039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.975813 5039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:47.975941 5076 mvcc.cc:205] Tried to move back new op lower bound from 7333736152940883968 to 7333736152866009088. Current Snapshot: MvccSnapshot[applied={T|T < 7333736152940883968}]
I20260926 22:48:47.975989 4958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.976127 4958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 536e4bb4b9f54115b6bf5639993e3b84, State: Initialized, Role: FOLLOWER
I20260926 22:48:47.976204 4958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } }
I20260926 22:48:47.976281 5077 mvcc.cc:205] Tried to move back new op lower bound from 7333736152940883968 to 7333736152866009088. Current Snapshot: MvccSnapshot[applied={T|T < 7333736152940883968}]
I20260926 22:48:47.976501 4958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:47.976604 4931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:47.977470 4931 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [term 1 FOLLOWER]: Refusing update from remote peer 10c7af9fb2ed4741a4a5bf5809ee2e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:47.977632 4932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "536e4bb4b9f54115b6bf5639993e3b84"
I20260926 22:48:47.978358 4932 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10c7af9fb2ed4741a4a5bf5809ee2e37 in term 1.
I20260926 22:48:47.978760 5073 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:47.980124 5075 catalog_manager.cc:1347] Generated new cluster ID: 3a011558f1224d08873ae2098c934980
I20260926 22:48:47.980185 5075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260926 22:48:47.980813 5093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:47.980909 5093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:47.981240 5082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.981379 5082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:47.981473 5082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c7af9fb2ed4741a4a5bf5809ee2e37. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.981565 5082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:47.981630 5082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:47.981712 5082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:47.981927 5095 mvcc.cc:205] Tried to move back new op lower bound from 7333736152940883968 to 7333736152866009088. Current Snapshot: MvccSnapshot[applied={T|T < 7333736152940883968}]
I20260926 22:48:48.046638 5053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c7af9fb2ed4741a4a5bf5809ee2e37" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 34557 } } peers { permanent_uuid: "536e4bb4b9f54115b6bf5639993e3b84" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 41391 } } peers { permanent_uuid: "d51fe638f1894428a627926517b868bc" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 36489 } } }
I20260926 22:48:48.046772 5053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:48.049273 5075 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:48.049994 5075 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:48.060961 5075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10c7af9fb2ed4741a4a5bf5809ee2e37: Generated new TSK 0
I20260926 22:48:48.061187 5075 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 22:48:48.081832 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.080745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.14.184.253:41391)
0926 22:48:48.080807 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.080809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.081150 (+ 341us) server_negotiation.cc:407] Connection header received
0926 22:48:48.081256 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.081258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.081269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.081301 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.081752 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37104: BlockingRecv error: recv got EOF from 127.0.0.1:37104 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260926 22:48:48.083228 4887 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.082311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.14.184.254:34557)
0926 22:48:48.082352 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.082354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.082656 (+ 302us) server_negotiation.cc:407] Connection header received
0926 22:48:48.082801 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.082802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.082813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.082843 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.083166 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48552: BlockingRecv error: recv got EOF from 127.0.0.1:48552 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 22:48:48.084010 5037 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.082540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36736 (local address 127.14.184.252:36489)
0926 22:48:48.082579 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.082581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.083502 (+ 921us) server_negotiation.cc:407] Connection header received
0926 22:48:48.083597 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.083598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.083608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.083638 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.083955 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36736: BlockingRecv error: recv got EOF from 127.0.0.1:36736 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 22:48:48.086320 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.085564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.14.184.253:41391)
0926 22:48:48.085601 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.085603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.085778 (+ 175us) server_negotiation.cc:407] Connection header received
0926 22:48:48.085900 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.085901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.085919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.085949 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.086273 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37114: BlockingRecv error: recv got EOF from 127.0.0.1:37114 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 22:48:48.089359 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.088546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.14.184.253:41391)
0926 22:48:48.088578 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.088580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.088851 (+ 271us) server_negotiation.cc:407] Connection header received
0926 22:48:48.088952 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.088953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.088963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.088993 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.089317 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37124: BlockingRecv error: recv got EOF from 127.0.0.1:37124 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:48:48.093132 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.092379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.14.184.253:41391)
0926 22:48:48.092412 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.092414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.092643 (+ 229us) server_negotiation.cc:407] Connection header received
0926 22:48:48.092737 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.092737 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.092748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.092776 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.093093 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37134: BlockingRecv error: recv got EOF from 127.0.0.1:37134 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:48:48.098994 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.098166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.14.184.253:41391)
0926 22:48:48.098200 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.098202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.098441 (+ 239us) server_negotiation.cc:407] Connection header received
0926 22:48:48.098541 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.098542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.098553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.098582 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.098950 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37136: BlockingRecv error: recv got EOF from 127.0.0.1:37136 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 22:48:48.108907 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.108056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.14.184.253:41391)
0926 22:48:48.108093 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.108095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.108346 (+ 251us) server_negotiation.cc:407] Connection header received
0926 22:48:48.108447 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.108448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.108459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.108489 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.108859 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37144: BlockingRecv error: recv got EOF from 127.0.0.1:37144 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 22:48:48.127261 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.126239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.14.184.253:41391)
0926 22:48:48.126281 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.126283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.126539 (+ 256us) server_negotiation.cc:407] Connection header received
0926 22:48:48.126645 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.126646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.126658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.126691 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.127201 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37150: BlockingRecv error: recv got EOF from 127.0.0.1:37150 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 22:48:48.161048 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.160359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.14.184.253:41391)
0926 22:48:48.160434 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.160435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.160535 (+ 100us) server_negotiation.cc:407] Connection header received
0926 22:48:48.160582 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.160582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.160592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.160655 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.160977 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37164: BlockingRecv error: recv got EOF from 127.0.0.1:37164 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 22:48:48.226990 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.226233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.14.184.253:41391)
0926 22:48:48.226318 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.226320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.226411 (+ 91us) server_negotiation.cc:407] Connection header received
0926 22:48:48.226454 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.226455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.226465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.226526 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.226931 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37174: BlockingRecv error: recv got EOF from 127.0.0.1:37174 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 22:48:48.356884 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.356070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.14.184.253:41391)
0926 22:48:48.356266 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.356268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.356284 (+ 16us) server_negotiation.cc:407] Connection header received
0926 22:48:48.356345 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.356346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.356358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.356428 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.356820 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37184: BlockingRecv error: recv got EOF from 127.0.0.1:37184 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260926 22:48:48.614712 4963 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:48.614084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.14.184.253:41391)
0926 22:48:48.614158 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:48.614160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:48.614172 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:48:48.614203 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:48.614205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:48.614214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:48.614285 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:48.614598 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37198: BlockingRecv error: recv got EOF from 127.0.0.1:37198 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 22:48:48.919641 5050 catalog_manager.cc:1259] Loaded cluster ID: 3a011558f1224d08873ae2098c934980
I20260926 22:48:48.919719 5050 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: loading cluster ID for follower catalog manager: success
I20260926 22:48:48.920521 5050 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: acquiring CA information for follower catalog manager: success
I20260926 22:48:48.920763 5050 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d51fe638f1894428a627926517b868bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:48:48.981382 5093 catalog_manager.cc:1259] Loaded cluster ID: 3a011558f1224d08873ae2098c934980
I20260926 22:48:48.981462 5093 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: loading cluster ID for follower catalog manager: success
I20260926 22:48:48.982295 5093 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: acquiring CA information for follower catalog manager: success
I20260926 22:48:48.982602 5093 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 536e4bb4b9f54115b6bf5639993e3b84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:48:49.130268 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:49.127841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.14.184.253:41391)
0926 22:48:49.127963 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:49.127965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:49.127985 (+ 20us) server_negotiation.cc:407] Connection header received
0926 22:48:49.128062 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:49.128063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:49.128111 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:49.128212 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:49.128342 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.128842 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.129564 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.129756 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.129810 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:49.129941 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:49.130045 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:49.130133 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:49.130182 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 22:48:49.130199 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260926 22:48:49.134554 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:49.132179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.14.184.252:36489)
0926 22:48:49.132316 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:49.132318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:49.132325 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:48:49.132356 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:49.132356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:49.132387 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:49.132475 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:49.132568 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.133046 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.133774 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.133971 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.134044 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:49.134203 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:49.134309 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:49.134388 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:49.134441 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 22:48:49.134461 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260926 22:48:49.139330 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:49.136219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.14.184.253:41391)
0926 22:48:49.136261 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:49.136263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:49.136379 (+ 116us) server_negotiation.cc:407] Connection header received
0926 22:48:49.136494 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:49.136496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:49.136507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:49.136571 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:49.136668 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.137393 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.138474 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.138785 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.138848 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:49.139045 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:49.139077 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:49.139139 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:49.139203 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 22:48:49.139223 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:48:49.139410 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:49.136390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.14.184.252:36489)
0926 22:48:49.136426 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:49.136428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:49.136541 (+ 113us) server_negotiation.cc:407] Connection header received
0926 22:48:49.136566 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:49.136567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:49.136577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:49.136621 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:49.136791 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.137249 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.138506 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.138760 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.138825 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:49.138999 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:49.139090 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:49.139144 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:49.139290 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 22:48:49.139314 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:48:49.139508 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:49.136181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48564 (local address 127.14.184.254:34557)
0926 22:48:49.136398 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:49.136400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:49.136409 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:49.136453 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:49.136454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:49.136500 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:49.136738 (+ 238us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:49.136779 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.137517 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.138324 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:49.138609 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:49.138868 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:49.138929 (+ 61us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:49.139035 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:49.139376 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:49.139395 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 22:48:49.139423 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":48,"threads_started":1}
I20260926 22:48:49.141276 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4815
I20260926 22:48:49.146727 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4890
I20260926 22:48:49.151297 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 4965
2026-09-26T22:48:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2836 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 22:48:49 dist-test-slave-3986 krb5kdc[5180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 22:48:49 dist-test-slave-3986 krb5kdc[5180](info): set up 2 sockets
Sep 26 22:48:49 dist-test-slave-3986 krb5kdc[5180](info): commencing operation
krb5kdc: starting...
W20260926 22:48:50.179242 15075 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/test-user.keytab.
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T22:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T22:48:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.184.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
Entry for principal kudu/127.14.184.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab.
I20260926 22:48:50.290256 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:39375
--webserver_interface=127.14.184.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:39375,127.14.184.253:37707,127.14.184.252:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 22:48:50.369051 5193 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 22:48:50.369292 5193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:50.369321 5193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:50.369364 5193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:50.370587 5193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:50.370668 5193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:50.370723 5193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:50.370765 5193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:50.370795 5193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:50.372289 5193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:39375,127.14.184.253:37707,127.14.184.252:39801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.254.keytab
--principal=kudu/127.14.184.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.254:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.184.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:50.372556 5193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:50.372779 5193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:50.377874 5198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:50.378015 5193 server_base.cc:1034] running on GCE node
W20260926 22:48:50.377849 5201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:50.377880 5199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:50.378331 5193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:50.378607 5193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:50.379766 5193 hybrid_clock.cc:648] HybridClock initialized: now 1790462930379743 us; error 37 us; skew 500 ppm
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:50.382169 5193 init.cc:377] Logged in from keytab as kudu/127.14.184.254@KRBTEST.COM (short username kudu)
I20260926 22:48:50.382570 5193 webserver.cc:460] Webserver started at http://127.14.184.254:33055/ using document root <none> and password file <none>
I20260926 22:48:50.382817 5193 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:50.382885 5193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:50.383035 5193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:50.384073 5193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data/instance:
uuid: "d371e614cd5743a8b41c645ad902c232"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "d1b2dfaedffc5cdde4c7324d816209e2"
server_key_iv: "4300c400213dc4227a62160aa54d7240"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.384446 5193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal/instance:
uuid: "d371e614cd5743a8b41c645ad902c232"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "d1b2dfaedffc5cdde4c7324d816209e2"
server_key_iv: "4300c400213dc4227a62160aa54d7240"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.385731 5193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:50.386422 5208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:50.386601 5193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:50.386725 5193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal
uuid: "d371e614cd5743a8b41c645ad902c232"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "d1b2dfaedffc5cdde4c7324d816209e2"
server_key_iv: "4300c400213dc4227a62160aa54d7240"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.386811 5193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:50.402187 5193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:50.404119 5193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:50.404281 5193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:50.408054 5193 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.254:39375
I20260926 22:48:50.408089 5260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.254:39375 every 8 connection(s)
I20260926 22:48:50.408437 5193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/data/info.pb
I20260926 22:48:50.409281 5261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:50.410754 5261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.414857 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 5193
I20260926 22:48:50.414958 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-0/wal/instance
I20260926 22:48:50.415179 15075 external_mini_cluster.cc:1428] Setting key fb98f584f5d676f7ceed1867ab4823c8
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:50.420877 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.411381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:39375 (local address 127.0.0.1:48618)
0926 22:48:50.411556 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.411598 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.412100 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.412383 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.412386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.412652 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.412792 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.412797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.413801 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.413802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.414471 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.414473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.415186 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.415187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.415286 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.415780 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.415787 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.417383 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.419486 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.419492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.419493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.419678 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.419958 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.419958 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.419959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.420033 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.420481 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.420485 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.420535 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.420599 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.420717 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260926 22:48:50.420994 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.411410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.14.184.254:39375)
0926 22:48:50.411565 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.411573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.411722 (+ 149us) server_negotiation.cc:407] Connection header received
0926 22:48:50.412253 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.412256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.412274 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.412379 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.412896 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.413690 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.414567 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.415080 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.415203 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.417510 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.417517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.417518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.417525 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.419406 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.419763 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.419767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.419767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.419882 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.420101 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.420102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.420102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.420403 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.420494 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.420668 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.420815 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260926 22:48:50.421680 5261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.421998 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.421884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:51954)
0926 22:48:50.421930 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.421970 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 22:48:50.422122 5211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
I20260926 22:48:50.422485 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.422400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:51970)
0926 22:48:50.422442 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.422465 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 22:48:50.422586 5261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:37707: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.184.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab.
Entry for principal kudu/127.14.184.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab.
I20260926 22:48:50.438536 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:37707
--webserver_interface=127.14.184.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab
--principal=kudu/127.14.184.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:39375,127.14.184.253:37707,127.14.184.252:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:48:50.471800 5261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } attempt: 1
I20260926 22:48:50.472242 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.472115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:51978)
0926 22:48:50.472174 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.472212 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:48:50.472586 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.472491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:51988)
0926 22:48:50.472536 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.472560 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:48:50.472666 5261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:37707: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
W20260926 22:48:50.519774 5268 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 22:48:50.519984 5268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:50.520000 5268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:50.520053 5268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:50.521183 5268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:50.521224 5268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:50.521237 5268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:50.521296 5268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:50.521328 5268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:50.522802 5268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:39375,127.14.184.253:37707,127.14.184.252:39801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.253.keytab
--principal=kudu/127.14.184.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.253:37707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.184.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:50.523046 5268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:50.523260 5268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:50.527858 5274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:50.527864 5276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:50.527853 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:50.528038 5268 server_base.cc:1034] running on GCE node
I20260926 22:48:50.528255 5268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:50.528589 5268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:50.529742 5268 hybrid_clock.cc:648] HybridClock initialized: now 1790462930529723 us; error 36 us; skew 500 ppm
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:50.532104 5268 init.cc:377] Logged in from keytab as kudu/127.14.184.253@KRBTEST.COM (short username kudu)
I20260926 22:48:50.532558 5268 webserver.cc:460] Webserver started at http://127.14.184.253:36451/ using document root <none> and password file <none>
I20260926 22:48:50.532763 5268 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:50.532805 5268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:50.532964 5268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:50.533932 5268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data/instance:
uuid: "64332ba44a7e4aab94b0b5e546d241e6"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "55fa5fadebbee311ed3296b4e7a84b80"
server_key_iv: "3f5c76c2c30b2afa331515bb0a89536c"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.534261 5268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal/instance:
uuid: "64332ba44a7e4aab94b0b5e546d241e6"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "55fa5fadebbee311ed3296b4e7a84b80"
server_key_iv: "3f5c76c2c30b2afa331515bb0a89536c"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.535633 5268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 22:48:50.536355 5283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:50.536634 5268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 22:48:50.536726 5268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal
uuid: "64332ba44a7e4aab94b0b5e546d241e6"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "55fa5fadebbee311ed3296b4e7a84b80"
server_key_iv: "3f5c76c2c30b2afa331515bb0a89536c"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.536798 5268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:50.540796 5261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } attempt: 2
I20260926 22:48:50.541154 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.541040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:51990)
0926 22:48:50.541094 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.541129 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 22:48:50.541518 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.541444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:52006)
0926 22:48:50.541481 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.541504 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 22:48:50.541613 5261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.253:37707: Network error: Client connection negotiation failed: client connection to 127.14.184.253:37707: connect: Connection refused (error 111)
I20260926 22:48:50.565951 5268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:50.567822 5268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:50.567983 5268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:50.572054 5268 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.253:37707
I20260926 22:48:50.572090 5335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.253:37707 every 8 connection(s)
I20260926 22:48:50.572577 5268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/data/info.pb
I20260926 22:48:50.573599 5336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:50.575203 5336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:50.582862 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 5268
I20260926 22:48:50.582966 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-1/wal/instance
I20260926 22:48:50.583179 15075 external_mini_cluster.cc:1428] Setting key 7fd07587c194c93bc718bc9ecd8261aa
I20260926 22:48:50.583581 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.575887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:39375 (local address 127.0.0.1:48632)
0926 22:48:50.576063 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.576096 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.576515 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.576744 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.576748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.576996 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.577178 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.577181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.577812 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.577812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.578555 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.578556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.579033 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.579034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.579169 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.579632 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.579646 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.580909 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.582227 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.582233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.582235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.582449 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.582627 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.582627 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.582627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.582778 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.583084 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.583089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.583196 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.583321 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.583434 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260926 22:48:50.583595 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.575878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48632 (local address 127.14.184.254:39375)
0926 22:48:50.575955 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.575957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.576174 (+ 217us) server_negotiation.cc:407] Connection header received
0926 22:48:50.576592 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.576593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.576601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.576671 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.577247 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.577725 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.578638 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.578953 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.579014 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.580989 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.580991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.580991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.580997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.582160 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.582519 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.582520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.582520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.582570 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.582844 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.582846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.582846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.582991 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.583082 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.583378 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.583467 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 22:48:50.584446 5336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.184.252@KRBTEST.COM; defaulting to no policy
I20260926 22:48:50.592046 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.584694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:52020)
0926 22:48:50.584756 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.584769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.584825 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.585027 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.585028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.585190 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.585255 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.585256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.586042 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.586047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.586937 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.586940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.587395 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.587396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.587472 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.587810 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.587816 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.589102 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.590943 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.590944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.590945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.591157 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.591377 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.591378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.591378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.591431 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.591714 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.591715 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.591750 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.591871 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.591973 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 22:48:50.592162 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.584739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.14.184.253:37707)
0926 22:48:50.584880 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.584884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.584902 (+ 18us) server_negotiation.cc:407] Connection header received
0926 22:48:50.584930 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.584931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.584941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.585001 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.585327 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.585961 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.587036 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.587319 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.587372 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.589195 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.589210 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.589210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.589216 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.590855 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.591247 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.591250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.591250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.591313 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.591499 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.591500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.591501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.591642 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.591734 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.591919 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.592041 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260926 22:48:50.592723 5336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.593031 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.592932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36810)
0926 22:48:50.592975 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.593014 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 22:48:50.593152 5287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.184.252@KRBTEST.COM" created.
I20260926 22:48:50.593587 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.593490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36822)
0926 22:48:50.593542 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.593565 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 22:48:50.593729 5336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:39801: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.184.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab.
Entry for principal kudu/127.14.184.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab.
I20260926 22:48:50.607129 15075 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmeYARH/build/release/bin/kudu
/tmp/dist-test-taskmeYARH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:39801
--webserver_interface=127.14.184.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab
--principal=kudu/127.14.184.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.184.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.184.254:39375,127.14.184.253:37707,127.14.184.252:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 22:48:50.632761 5261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } attempt: 3
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:48:50.639632 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.633193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.14.184.253:37707)
0926 22:48:50.633267 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.633269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.633281 (+ 12us) server_negotiation.cc:407] Connection header received
0926 22:48:50.633320 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.633321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.633336 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.633402 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.633728 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.634222 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.634985 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.635212 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.635305 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.637077 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.637078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.637079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.637086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.638453 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.638867 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.638869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.638869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.638939 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.639138 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.639139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.639139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.639246 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.639325 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.639468 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.639539 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 22:48:50.639632 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.633114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:52032)
0926 22:48:50.633183 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.633194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.633257 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.633409 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.633410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.633576 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.633642 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.633644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.634316 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.634319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.634906 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.634909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.635303 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.635305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.635381 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.635834 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.635839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.636992 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.638537 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.638538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.638539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.638786 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.639015 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.639016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.639016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.639078 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.639316 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.639317 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.639361 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.639445 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.639536 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 22:48:50.640022 5261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.640429 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.640293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36830)
0926 22:48:50.640358 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.640396 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 22:48:50.640820 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.640729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36834)
0926 22:48:50.640780 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.640804 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 22:48:50.640923 5261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:39801: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
I20260926 22:48:50.642927 5336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } attempt: 1
I20260926 22:48:50.643242 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.643146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36850)
0926 22:48:50.643193 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.643218 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 22:48:50.643579 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.643495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36858)
0926 22:48:50.643535 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.643557 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 22:48:50.643666 5336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:39801: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
I20260926 22:48:50.672060 5261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } attempt: 1
I20260926 22:48:50.672538 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.672414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36870)
0926 22:48:50.672476 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.672513 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 22:48:50.672928 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.672852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36874)
0926 22:48:50.672890 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.672913 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:48:50.673045 5261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:39801: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
W20260926 22:48:50.687135 5343 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 22:48:50.687325 5343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 22:48:50.687343 5343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 22:48:50.687363 5343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 22:48:50.688524 5343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 22:48:50.688565 5343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 22:48:50.688580 5343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 22:48:50.688591 5343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 22:48:50.688649 5343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 22:48:50.690176 5343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.184.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.184.254:39375,127.14.184.253:37707,127.14.184.252:39801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/krb5kdc/kudu_127.14.184.252.keytab
--principal=kudu/127.14.184.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.184.252:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.184.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 21:54:42 UTC on ccycloud.quasar-awvddz.root.comops.site
build id 4675485
I20260926 22:48:50.690452 5343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 22:48:50.690754 5343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 22:48:50.695333 5349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:50.695358 5348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 22:48:50.695333 5351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 22:48:50.695555 5343 server_base.cc:1034] running on GCE node
I20260926 22:48:50.695751 5343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 22:48:50.696033 5343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 22:48:50.697177 5343 hybrid_clock.cc:648] HybridClock initialized: now 1790462930697162 us; error 29 us; skew 500 ppm
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 22:48:50.699501 5343 init.cc:377] Logged in from keytab as kudu/127.14.184.252@KRBTEST.COM (short username kudu)
I20260926 22:48:50.699952 5343 webserver.cc:460] Webserver started at http://127.14.184.252:42729/ using document root <none> and password file <none>
I20260926 22:48:50.700186 5343 fs_manager.cc:359] Metadata directory not provided
I20260926 22:48:50.700255 5343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 22:48:50.700410 5343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 22:48:50.701406 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data/instance:
uuid: "6db9a8c367b1488293f3d7cf128547ad"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "2f44465f590839ae43250ddc43310f82"
server_key_iv: "dfcdbccf1ac77504c3bbd106dd2b61d5"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.701754 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal/instance:
uuid: "6db9a8c367b1488293f3d7cf128547ad"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "2f44465f590839ae43250ddc43310f82"
server_key_iv: "dfcdbccf1ac77504c3bbd106dd2b61d5"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.703106 5343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 22:48:50.703835 5358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 22:48:50.704017 5343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 22:48:50.704115 5343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data,/tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal
uuid: "6db9a8c367b1488293f3d7cf128547ad"
format_stamp: "Formatted at 2026-09-26 22:48:50 on dist-test-slave-3986"
server_key: "2f44465f590839ae43250ddc43310f82"
server_key_iv: "dfcdbccf1ac77504c3bbd106dd2b61d5"
server_key_version: "encryptionkey@0"
I20260926 22:48:50.704190 5343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 22:48:50.711797 5336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } attempt: 2
I20260926 22:48:50.712133 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.712021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36890)
0926 22:48:50.712075 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.712108 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 22:48:50.712433 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.712343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36896)
0926 22:48:50.712385 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.712408 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 22:48:50.712524 5336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.184.252:39801: Network error: Client connection negotiation failed: client connection to 127.14.184.252:39801: connect: Connection refused (error 111)
I20260926 22:48:50.730063 5343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 22:48:50.731868 5343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 22:48:50.732024 5343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 22:48:50.735689 5343 rpc_server.cc:307] RPC server started. Bound to: 127.14.184.252:39801
I20260926 22:48:50.735719 5410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.184.252:39801 every 8 connection(s)
I20260926 22:48:50.736066 5343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/data/info.pb
I20260926 22:48:50.736852 5411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 22:48:50.738209 5411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.741644 15075 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmeYARH/build/release/bin/kudu as pid 5343
I20260926 22:48:50.741755 15075 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmeYARH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790462668484683-15075-0/minicluster-data/master-2/wal/instance
I20260926 22:48:50.741973 15075 external_mini_cluster.cc:1428] Setting key 056e6c7573221384690f27f6691b25a8
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:50.747236 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.738926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.14.184.254:39375)
0926 22:48:50.738979 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.738981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.739258 (+ 277us) server_negotiation.cc:407] Connection header received
0926 22:48:50.739760 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.739761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.739771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.739822 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.740418 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.740859 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.741996 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.742223 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.742306 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.744389 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.744390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.744390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.744397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.745877 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.746264 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.746265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.746266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.746324 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.746554 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.746555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.746556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.746730 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.746815 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.747046 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.747145 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 22:48:50.747327 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.738914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.254:39375 (local address 127.0.0.1:48638)
0926 22:48:50.739115 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.739157 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.739669 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.739877 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.739880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.740100 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.740342 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.740347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.740927 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.740928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.741919 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.741922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.742310 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.742312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.742473 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.743042 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.743057 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.744311 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.745945 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.745952 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.745954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.746192 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.746395 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.746396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.746396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.746488 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.746803 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.746807 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.746913 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.746995 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.747098 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:50.747923 5411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.748203 5261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } attempt: 2
I20260926 22:48:50.751866 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.742336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.14.184.252:39801)
0926 22:48:50.742487 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.742492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.742636 (+ 144us) server_negotiation.cc:407] Connection header received
0926 22:48:50.742659 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.742661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.742711 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.742780 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.743390 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.744082 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.745128 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.745493 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.745547 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.748309 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.748315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.748316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.748321 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.750032 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.750415 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.750418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.750418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.750476 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.750723 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.750723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.750724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.750888 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.751614 (+ 726us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.751666 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.751752 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.254@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:50.755878 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.748172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.253:37707 (local address 127.0.0.1:52040)
0926 22:48:50.748212 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.748222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.748252 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.748409 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.748410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.748579 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.748630 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.748631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.749533 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.749534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.750281 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.750283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.750778 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.750780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.750849 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.751248 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.751253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.752688 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.754483 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.754484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.754485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.754789 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.755222 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.755223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.755223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.755297 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.755565 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.755566 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.755610 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.755711 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.755787 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 22:48:50.755927 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.748246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.14.184.253:37707)
0926 22:48:50.748293 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.748295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.748304 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:50.748329 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.748330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.748340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.748397 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.748734 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.749445 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.750367 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.750706 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.750780 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.752779 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.752780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.752780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.752787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.754405 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.755085 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.755087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.755087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.755147 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.755371 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.755372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.755372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.755496 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.755562 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.755747 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.755835 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 22:48:50.756201 5411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } has no permanent_uuid. Determining permanent_uuid...
I20260926 22:48:50.757030 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.748409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36914)
0926 22:48:50.748451 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.748461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.748491 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.748886 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.748886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.749020 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.749051 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.749052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.749997 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.749998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.750704 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.750706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.751159 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.751161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.751222 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.751570 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.751573 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.753261 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.755691 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.755692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.755692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.755896 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.756380 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.756380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.756381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.756447 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.756720 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.756721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.756768 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.756835 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.756921 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 22:48:50.757665 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.748608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36914 (local address 127.14.184.252:39801)
0926 22:48:50.748755 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.748756 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.748763 (+ 7us) server_negotiation.cc:407] Connection header received
0926 22:48:50.748781 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.748781 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.748796 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.748869 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.749132 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.749908 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.750810 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.751073 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.751365 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.753357 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.753358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.753359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.753363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.755618 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.755980 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.755981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.755981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.756044 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.756523 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.756524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.756525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.756663 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.756727 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.757417 (+ 690us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.757541 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.254@KRBTEST.COM
I20260926 22:48:50.759666 5261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232: Bootstrap starting.
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.252@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:50.760422 5261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:50.760774 5261 log.cc:824] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:50.761615 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.753233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.14.184.254:39375)
0926 22:48:50.753317 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.753319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.753328 (+ 9us) server_negotiation.cc:407] Connection header received
0926 22:48:50.753349 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.753350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.753360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.753422 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.753978 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.754658 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.755642 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.755932 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.755988 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.758841 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.758842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.758842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.758849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.760297 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.760715 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.760716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.760717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.760772 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.760983 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.760985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.760985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.761114 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.761240 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.761457 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.761526 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 22:48:50.761629 5261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232: No bootstrap required, opened a new log
I20260926 22:48:50.763659 5261 raft_consensus.cc:348] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.763829 5261 raft_consensus.cc:729] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d371e614cd5743a8b41c645ad902c232, State: Initialized, Role: FOLLOWER
I20260926 22:48:50.763916 5261 consensus_queue.cc:260] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.764194 5418 sys_catalog.cc:455] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.764258 5261 sys_catalog.cc:564] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:50.764343 5418 sys_catalog.cc:458] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:50.764443 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.756442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36928)
0926 22:48:50.756478 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.756487 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.756523 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.756627 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.756628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.756761 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.756783 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.756784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.757365 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.757366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.757942 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.757944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.758299 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.758300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.758347 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.758749 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.758760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.760159 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.763477 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.763478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.763479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.763685 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.763895 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.763895 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.763895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.763940 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.764187 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.764187 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.764216 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.764266 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.764339 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 22:48:50.764729 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.756469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.14.184.252:39801)
0926 22:48:50.756499 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.756500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.756510 (+ 10us) server_negotiation.cc:407] Connection header received
0926 22:48:50.756576 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.756576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.756584 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.756627 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.756854 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.757286 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.758016 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.758212 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.758298 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.760244 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.760245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.760245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.760251 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.763400 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.763775 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.763776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.763776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.763838 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.763994 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.763994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.763994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.764120 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.764169 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.764592 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.764659 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:48:50.766212 5429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 22:48:50.766222 5411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: Bootstrap starting.
W20260926 22:48:50.766269 5429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.184.253@KRBTEST.COM
I20260926 22:48:50.766997 5411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:50.767287 5411 log.cc:824] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:50.767011 5193 master_runner.cc:186] Error getting master registration for 127.14.184.253:37707: OK, instance_id { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" instance_seqno: 1790462930570834 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 22:48:50.768895 5193 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 22:48:50.769570 5411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: No bootstrap required, opened a new log
I20260926 22:48:50.770375 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.762226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52050 (local address 127.14.184.253:37707)
0926 22:48:50.762275 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.762277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.762285 (+ 8us) server_negotiation.cc:407] Connection header received
0926 22:48:50.762306 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.762307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.762315 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.762364 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.762864 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.763292 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.764429 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.764725 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.764785 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.767435 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.767436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.767437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.767443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.769124 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.769505 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.769506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.769506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.769561 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.769832 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.769833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.769834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.769943 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.770133 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.770178 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.770279 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 22:48:50.771600 15075 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 22:48:50.771663 5411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.771816 5411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db9a8c367b1488293f3d7cf128547ad, State: Initialized, Role: FOLLOWER
I20260926 22:48:50.771901 5411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.772173 5411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:50.772176 5432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.772390 5432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:50.772579 5235 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d371e614cd5743a8b41c645ad902c232"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48654
I20260926 22:48:50.772713 5235 raft_consensus.cc:482] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 22:48:50.772789 5235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:50.773725 5235 raft_consensus.cc:504] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.774004 5235 leader_election.cc:288] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [CANDIDATE]: Term 1 election: Requested vote from peers 64332ba44a7e4aab94b0b5e546d241e6 (127.14.184.253:37707), 6db9a8c367b1488293f3d7cf128547ad (127.14.184.252:39801)
I20260926 22:48:50.774314 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d371e614cd5743a8b41c645ad902c232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6db9a8c367b1488293f3d7cf128547ad"
I20260926 22:48:50.774417 5385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [term 0 FOLLOWER]: Advancing to term 1
W20260926 22:48:50.774457 5443 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:50.774488 5443 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:50.775512 5385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d371e614cd5743a8b41c645ad902c232 in term 1.
I20260926 22:48:50.775714 5212 leader_election.cc:302] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6db9a8c367b1488293f3d7cf128547ad, d371e614cd5743a8b41c645ad902c232; no voters:
I20260926 22:48:50.775844 5418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 22:48:50.775981 5418 raft_consensus.cc:686] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [term 1 LEADER]: Becoming Leader. State: Replica: d371e614cd5743a8b41c645ad902c232, State: Running, Role: LEADER
I20260926 22:48:50.776080 5418 consensus_queue.cc:237] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.776502 5418 sys_catalog.cc:455] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d371e614cd5743a8b41c645ad902c232. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.776595 5418 sys_catalog.cc:458] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:50.777311 5343 master_runner.cc:186] Error getting master registration for 127.14.184.253:37707: OK, instance_id { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" instance_seqno: 1790462930570834 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 22:48:50.779600 5343 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 22:48:50.779749 5452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 22:48:50.779947 5452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 22:48:50.782609 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.775112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.14.184.254:39375)
0926 22:48:50.775149 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.775151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.775418 (+ 267us) server_negotiation.cc:407] Connection header received
0926 22:48:50.775519 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.775520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.775531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.775561 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.776332 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.777043 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.778034 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.778320 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.778361 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.779889 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.779890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.779890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.779898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.781488 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.781829 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.781830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.781844 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.781899 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.782065 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.782066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.782066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.782211 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.782313 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.782422 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.782490 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 22:48:50.789299 5385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [term 1 FOLLOWER]: Refusing update from remote peer d371e614cd5743a8b41c645ad902c232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:50.789659 5418 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:50.790560 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.780518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.14.184.253:37707)
0926 22:48:50.780555 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.780557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.782661 (+ 2104us) server_negotiation.cc:407] Connection header received
0926 22:48:50.782701 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.782702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.782713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.782763 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.783420 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.784074 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.785064 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.785363 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.785417 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.787422 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.787423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.787423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.787430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.789064 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.789482 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.789483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.789484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.789557 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.789812 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.789813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.789813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.789964 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.790050 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.790392 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.790474 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:50.791425 5432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader d371e614cd5743a8b41c645ad902c232. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.791553 5432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:50.794255 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.779366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.14.184.252:39801)
0926 22:48:50.779409 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.779410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.780282 (+ 872us) server_negotiation.cc:407] Connection header received
0926 22:48:50.780724 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.780725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.780735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.780765 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.785383 (+ 4618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.786033 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.787302 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.787590 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.787635 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.791341 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.791342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.791343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.791350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.792933 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.793402 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.793403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.793404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.793458 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.793680 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.793681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.793681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.793819 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.793907 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.794083 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.794158 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 22:48:50.803648 5336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } attempt: 3
I20260926 22:48:50.803684 5450 sys_catalog.cc:455] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.803774 5450 sys_catalog.cc:458] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:50.804276 5432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.804358 5432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 22:48:50 dist-test-slave-3986 krb5kdc[5180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790462930, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.184.253@KRBTEST.COM for kudu/127.14.184.252@KRBTEST.COM
I20260926 22:48:50.811141 5452 catalog_manager.cc:1347] Generated new cluster ID: 8f74eec5e6524e9b814da06b256bddaa
I20260926 22:48:50.811193 5452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 22:48:50.811460 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.803912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.184.252:39801 (local address 127.0.0.1:36944)
0926 22:48:50.803963 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 22:48:50.803975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 22:48:50.804029 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 22:48:50.804201 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 22:48:50.804201 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 22:48:50.804363 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 22:48:50.804403 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.804405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.805351 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.805352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.806148 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 22:48:50.806150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.806601 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.806602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 22:48:50.806687 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.807087 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 22:48:50.807092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 22:48:50.808580 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 22:48:50.810428 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.810429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.810430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.810595 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.810814 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 22:48:50.810814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 22:48:50.810815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 22:48:50.810885 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 22:48:50.811166 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 22:48:50.811167 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 22:48:50.811213 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 22:48:50.811277 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 22:48:50.811369 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 22:48:50.811759 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:50.803957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.14.184.252:39801)
0926 22:48:50.803997 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.803999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.804010 (+ 11us) server_negotiation.cc:407] Connection header received
0926 22:48:50.804100 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.804101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.804109 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.804163 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.804479 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.805264 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.806239 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:50.806538 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:50.806601 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:50.808651 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 22:48:50.808653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 22:48:50.808653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 22:48:50.808660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 22:48:50.810360 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.810686 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.810688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.810688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.810749 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 22:48:50.810944 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 22:48:50.810945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 22:48:50.810946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 22:48:50.811079 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 22:48:50.811138 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:50.811610 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 22:48:50.811671 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 22:48:50.816121 5336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: Bootstrap starting.
I20260926 22:48:50.816845 5336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: Neither blocks nor log segments found. Creating new log.
I20260926 22:48:50.817126 5336 log.cc:824] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: Log is configured to *not* fsync() on all Append() calls
I20260926 22:48:50.836880 5336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: No bootstrap required, opened a new log
I20260926 22:48:50.838629 5452 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 22:48:50.839205 5336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.839344 5336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64332ba44a7e4aab94b0b5e546d241e6, State: Initialized, Role: FOLLOWER
I20260926 22:48:50.839428 5336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } }
I20260926 22:48:50.839807 5452 catalog_manager.cc:1504] Loading token signing keys...
I20260926 22:48:50.839834 5336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 22:48:50.841022 5310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d371e614cd5743a8b41c645ad902c232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "64332ba44a7e4aab94b0b5e546d241e6"
I20260926 22:48:50.841153 5309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 22:48:50.842809 5309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [term 1 FOLLOWER]: Refusing update from remote peer d371e614cd5743a8b41c645ad902c232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 22:48:50.843101 5450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 22:48:50.845531 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.845614 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:50.845666 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d371e614cd5743a8b41c645ad902c232. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.845722 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 22:48:50.845757 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.845813 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 22:48:50.846397 5472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 22:48:50.846455 5472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 22:48:50.847316 5310 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d371e614cd5743a8b41c645ad902c232 for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260926 22:48:50.848084 5450 sys_catalog.cc:455] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d371e614cd5743a8b41c645ad902c232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d371e614cd5743a8b41c645ad902c232" member_type: VOTER last_known_addr { host: "127.14.184.254" port: 39375 } } peers { permanent_uuid: "64332ba44a7e4aab94b0b5e546d241e6" member_type: VOTER last_known_addr { host: "127.14.184.253" port: 37707 } } peers { permanent_uuid: "6db9a8c367b1488293f3d7cf128547ad" member_type: VOTER last_known_addr { host: "127.14.184.252" port: 39801 } } }
I20260926 22:48:50.848163 5450 sys_catalog.cc:458] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232 [sys.catalog]: This master's current role is: LEADER
I20260926 22:48:50.848171 5474 mvcc.cc:205] Tried to move back new op lower bound from 7333736164710137856 to 7333736164459610112. Current Snapshot: MvccSnapshot[applied={T|T < 7333736164510007296}]
I20260926 22:48:50.849252 5452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d371e614cd5743a8b41c645ad902c232: Generated new TSK 0
I20260926 22:48:50.849442 5452 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 22:48:50.878795 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.877337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.14.184.253:37707)
0926 22:48:50.877375 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.877377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.878075 (+ 698us) server_negotiation.cc:407] Connection header received
0926 22:48:50.878188 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.878189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.878200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.878231 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.878722 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52070: BlockingRecv error: recv got EOF from 127.0.0.1:52070 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 22:48:50.879556 5264 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.877118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48666 (local address 127.14.184.254:39375)
0926 22:48:50.877165 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.877167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.879050 (+ 1883us) server_negotiation.cc:407] Connection header received
0926 22:48:50.879151 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.879152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.879163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.879192 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.879496 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48666: BlockingRecv error: recv got EOF from 127.0.0.1:48666 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 22:48:50.880249 5415 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.877522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.14.184.252:39801)
0926 22:48:50.877559 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.877560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.879781 (+ 2221us) server_negotiation.cc:407] Connection header received
0926 22:48:50.879873 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.879874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.879884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.879912 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.880194 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36954: BlockingRecv error: recv got EOF from 127.0.0.1:36954 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 22:48:50.882581 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.881416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52078 (local address 127.14.184.253:37707)
0926 22:48:50.881451 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.881453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.882043 (+ 590us) server_negotiation.cc:407] Connection header received
0926 22:48:50.882172 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.882173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.882191 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.882222 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.882538 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52078: BlockingRecv error: recv got EOF from 127.0.0.1:52078 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 22:48:50.885530 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.884786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.14.184.253:37707)
0926 22:48:50.884820 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.884822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.885057 (+ 235us) server_negotiation.cc:407] Connection header received
0926 22:48:50.885153 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.885154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.885164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.885193 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.885492 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52082: BlockingRecv error: recv got EOF from 127.0.0.1:52082 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 22:48:50.889369 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.888542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.14.184.253:37707)
0926 22:48:50.888573 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.888575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.888868 (+ 293us) server_negotiation.cc:407] Connection header received
0926 22:48:50.888962 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.888963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.888991 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.889021 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.889330 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52094: BlockingRecv error: recv got EOF from 127.0.0.1:52094 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:48:50.895200 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.894374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.14.184.253:37707)
0926 22:48:50.894406 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.894408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.894717 (+ 309us) server_negotiation.cc:407] Connection header received
0926 22:48:50.894819 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.894820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.894829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.894858 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.895161 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52096: BlockingRecv error: recv got EOF from 127.0.0.1:52096 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 22:48:50.905025 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.904226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.14.184.253:37707)
0926 22:48:50.904262 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.904264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.904507 (+ 243us) server_negotiation.cc:407] Connection header received
0926 22:48:50.904604 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.904605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.904615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.904660 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.904984 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52112: BlockingRecv error: recv got EOF from 127.0.0.1:52112 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 22:48:50.923199 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.922229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.14.184.253:37707)
0926 22:48:50.922271 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.922273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.922548 (+ 275us) server_negotiation.cc:407] Connection header received
0926 22:48:50.922695 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.922697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.922711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.922743 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.923144 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52124: BlockingRecv error: recv got EOF from 127.0.0.1:52124 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 22:48:50.957302 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:50.956510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.14.184.253:37707)
0926 22:48:50.956556 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:50.956558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:50.956704 (+ 146us) server_negotiation.cc:407] Connection header received
0926 22:48:50.956783 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:50.956784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:50.956795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:50.956864 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:50.957228 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52138: BlockingRecv error: recv got EOF from 127.0.0.1:52138 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260926 22:48:51.023136 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:51.022436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.14.184.253:37707)
0926 22:48:51.022524 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.022526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.022619 (+ 93us) server_negotiation.cc:407] Connection header received
0926 22:48:51.022704 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.022705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.022717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.022782 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:51.023089 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52146: BlockingRecv error: recv got EOF from 127.0.0.1:52146 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260926 22:48:51.153137 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:51.152346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.14.184.253:37707)
0926 22:48:51.152432 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.152434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.152462 (+ 28us) server_negotiation.cc:407] Connection header received
0926 22:48:51.152597 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.152598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.152618 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.152710 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:51.153063 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52148: BlockingRecv error: recv got EOF from 127.0.0.1:52148 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260926 22:48:51.410964 5340 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 22:48:51.410260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.14.184.253:37707)
0926 22:48:51.410333 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.410335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.410349 (+ 14us) server_negotiation.cc:407] Connection header received
0926 22:48:51.410441 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.410443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.410461 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.410526 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 22:48:51.410921 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52154: BlockingRecv error: recv got EOF from 127.0.0.1:52154 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 22:48:51.774941 5443 catalog_manager.cc:1259] Loaded cluster ID: 8f74eec5e6524e9b814da06b256bddaa
I20260926 22:48:51.775015 5443 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: loading cluster ID for follower catalog manager: success
I20260926 22:48:51.775822 5443 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: acquiring CA information for follower catalog manager: success
I20260926 22:48:51.776146 5443 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6db9a8c367b1488293f3d7cf128547ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:48:51.846891 5472 catalog_manager.cc:1259] Loaded cluster ID: 8f74eec5e6524e9b814da06b256bddaa
I20260926 22:48:51.846978 5472 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: loading cluster ID for follower catalog manager: success
I20260926 22:48:51.847767 5472 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: acquiring CA information for follower catalog manager: success
I20260926 22:48:51.848064 5472 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64332ba44a7e4aab94b0b5e546d241e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 22:48:51.927172 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:51.924292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.14.184.253:37707)
0926 22:48:51.924421 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.924424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.924486 (+ 62us) server_negotiation.cc:407] Connection header received
0926 22:48:51.924597 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.924598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.924672 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.924767 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:51.924931 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.925438 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.926235 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.926475 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.926568 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:51.926772 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:51.926923 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:51.927013 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:51.927052 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 22:48:51.927076 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260926 22:48:51.930920 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:51.928834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.14.184.252:39801)
0926 22:48:51.928928 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.928930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.928959 (+ 29us) server_negotiation.cc:407] Connection header received
0926 22:48:51.929015 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.929015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.929047 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.929104 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:51.929167 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.929609 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.930234 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.930410 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.930457 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:51.930577 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:51.930659 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:51.930768 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:51.930796 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 22:48:51.930812 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":46,"threads_started":1}
I20260926 22:48:51.935012 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:51.932306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52162 (local address 127.14.184.253:37707)
0926 22:48:51.932343 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.932344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.932489 (+ 145us) server_negotiation.cc:407] Connection header received
0926 22:48:51.932513 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.932514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.932525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.932578 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:51.932723 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.933277 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.934263 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.934548 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.934634 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:51.934786 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:51.934814 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:51.934861 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:51.934921 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 22:48:51.934939 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:48:51.935317 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:51.932412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.14.184.252:39801)
0926 22:48:51.932447 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.932449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.932583 (+ 134us) server_negotiation.cc:407] Connection header received
0926 22:48:51.932687 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.932688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.932696 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.932744 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:51.932883 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.933519 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.934508 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.934839 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.934900 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:51.935053 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:51.935080 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:51.935151 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:51.935235 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 22:48:51.935258 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 22:48:51.935742 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 22:48:51.932369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.14.184.254:39375)
0926 22:48:51.932575 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 22:48:51.932577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 22:48:51.932735 (+ 158us) server_negotiation.cc:407] Connection header received
0926 22:48:51.932786 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 22:48:51.932787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 22:48:51.932833 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 22:48:51.932902 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 22:48:51.933065 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.933740 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.934686 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 22:48:51.935100 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 22:48:51.935159 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 22:48:51.935384 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 22:48:51.935486 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 22:48:51.935552 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 22:48:51.935618 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 22:48:51.935641 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":53,"threads_started":1}
I20260926 22:48:51.937496 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 5193
I20260926 22:48:51.943049 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 5268
I20260926 22:48:51.948033 15075 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmeYARH/build/release/bin/kudu with pid 5343
2026-09-26T22:48:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2796 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5632 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (263470 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST